Public note
Omi Is Building an Open-Source AI Memory Stack Across Wearables, Desktop, and Mobile
Omi is an open-source AI memory platform by BasedHardware that captures conversations and screen activity across desktop, mobile, and wearables, providing real-time transcription, summaries, action items, and an AI chat interface.
BasedHardware’s repository is not just a single app repo. It is the codebase behind Omi’s cross-device “second brain” product, with clients for desktop, phone, firmware, backend services, and an extensible plugin model.
Lead
Omi is an open-source AI memory platform by BasedHardware. According to the repository’s README, the project captures screen activity and conversations, performs real-time transcription, generates summaries and action items, and offers an AI chat layer that can remember what the user has seen and heard. The project says it works across desktop, phone, and wearables, making the repository notable less as a simple app and more as a full product stack under active development.
That framing matters. Many “AI memory” tools are delivered as cloud services with limited transparency into how the full stack works. Omi, by contrast, exposes a wide swath of the implementation in one public repository: client apps, firmware, backend services, documentation, plugins, SDKs, and web components. For developers, that makes the repo useful not only as an application source tree, but also as a reference architecture for building persistent multimodal memory products.
At a Glance
- Project: Omi
- Owner: BasedHardware
- Repository:
BasedHardware/omi - What it is: An open-source AI memory platform spanning wearable, desktop, mobile, backend, and plugin components
- Core promise: Capture conversations and screen activity, transcribe them, summarize them, and make them queryable through AI chat
- License: MIT
- Quick-start emphasis: The README highlights a macOS-first path for spinning up the desktop client against the hosted backend
What Happened
The repository presents Omi as “a 2nd brain you trust more than your 1st,” with the core product description centered on memory capture and retrieval. In the README, BasedHardware says Omi can capture a user’s screen and conversations, generate transcripts in real time, create summaries and action items, and provide an AI chat interface that remembers prior context.
The same README also reveals the project’s scope. Top-level directories include:
desktop/for the macOS applicationapp/for the mobile applicationbackend/for the API and processing servicesomi/for firmwareomiGlass/for the glasses-related hardware pathplugins/,sdks/, andweb/for ecosystem and developer surfaces
That breadth makes Omi more ambitious than a transcription utility. It is trying to unify capture, processing, memory, and post-conversation actions across multiple device types.
Key Facts / Comparison
| Area | What the sources say |
|---|---|
| Core function | Omi captures screen activity and conversations, transcribes in real time, generates summaries and action items, and offers AI chat over remembered context. |
| Device coverage | The README says Omi works on desktop, phone, and wearables. |
| Open-source status | The repository describes the project as fully open source and MIT licensed. |
| Desktop stack | The architecture section lists the macOS app as Swift/SwiftUI with a Rust backend component. |
| Mobile stack | The mobile app is listed as Flutter for iOS and Android. |
| Backend stack | The backend is described as Python with FastAPI and Firebase. |
| Firmware | The firmware path is listed as nRF, Zephyr, and C. |
| Ecosystem | Official help documentation describes Omi plugins/apps as modular extensions that can analyze conversations and connect to external services. |
Background and Context
Omi sits in a fast-growing category sometimes described as AI note-taking, personal memory, or second-brain software. What distinguishes this repo is that it spans both the user-facing product and a significant amount of the infrastructure behind it.
The README’s architecture sketch shows three major capture surfaces—wearable hardware, macOS desktop, and mobile—feeding into an Omi backend. That backend is described as combining REST and websocket services with components for speech processing, caching, storage, and LLM-driven AI features. The exact production deployment model is not fully documented in the repository landing page, but the high-level system design is explicit enough to show that Omi is intended as a multi-device pipeline, not a single recorder app.
Omi’s public web materials reinforce that positioning. The product site describes the app as turning conversations and meetings into searchable transcripts, summaries, action items, and memories. Official docs further note that users can build Omi apps/plugins and that conversations are stored in cloud infrastructure, with deletion controls exposed in the app settings.
Why This Matters
For developers and product teams, Omi is interesting for three reasons.
First, it treats memory as a system problem rather than just an LLM feature. The repo combines capture hardware, cross-platform apps, backend processing, and retrieval UX in one place.
Second, it pushes beyond passive transcription. Official materials repeatedly emphasize action items, tasks, and plugin-based integration, suggesting that Omi is trying to connect memory capture to execution workflows.
Third, the repository gives outsiders a rare look at the implementation layers behind a wearable-plus-desktop AI product. Even if a team does not want to use Omi directly, the codebase offers a concrete example of how multimodal capture products can be organized.
Insight and Industry Analysis
The strongest takeaway from the repository is that Omi is aiming for a persistent personal context layer rather than a single-purpose meeting assistant.
That is a strategically important distinction. A meeting summarizer usually starts and stops around a single session. Omi’s pitch is broader: let the system continuously gather signals from conversations and screens, then make that context queryable later. In industry terms, that moves the project closer to a memory operating layer for knowledge work.
The open-source angle is also notable. AI memory products often raise questions around trust, retention, portability, and vendor lock-in. Omi does not solve all of those concerns by default, but publishing the code, documenting the stack, and exposing plugin surfaces gives developers more room to inspect, self-host pieces, or build adjacent tools than they would get from a black-box product.
Strengths, Limitations, and Open Questions
Strengths
- Broad scope across hardware, desktop, mobile, backend, and extensions
- Clear product thesis around capture, summarization, memory, and actionability
- Open-source MIT licensing lowers barriers for inspection and experimentation
- Plugin/app model suggests room for workflow-specific extensions
Limitations
- The public repository landing page gives only a high-level view of production architecture and operational tradeoffs
- The quick-start path is heavily oriented around macOS, which may narrow initial contributor accessibility
- Some core services depend on external infrastructure and vendors, which means “open source” does not automatically equal fully local or fully self-contained operation
Open Questions
- How much of the end-to-end experience can be run locally versus through hosted services?
- What governance and consent patterns are most realistic for wearable and ambient capture use cases?
- How will the plugin ecosystem evolve as third-party integrations grow?
Technical Deep Dive
The most useful technical clue in the README is the architecture overview.
At the edge, Omi supports multiple input surfaces:
- Wearable device path over BLE
- macOS application over HTTPS/WebSocket flows
- Mobile application as another capture and access surface
Inside the backend, the diagram names several processing and infrastructure components:
- Listen (REST)
- Pusher (WS)
- VAD
- Diarizer
- Deepgram for speech-to-text
- Firestore for storage
- Redis for caching
- LLMs for higher-level AI behavior
From a systems perspective, this suggests a layered pipeline:
- Capture audio and possibly screen context on device.
- Stream or upload data into backend services.
- Run transcription and speaker/session processing.
- Persist outputs into memory/storage layers.
- Expose those memories to chat, summaries, tasks, and plugins.
The repo also points to different implementation languages for different layers—Swift/SwiftUI and Rust on desktop, Flutter on mobile, Python/FastAPI in the backend, and embedded C/Zephyr for firmware. That mix reflects the practical reality of multimodal AI products: no single stack cleanly covers firmware, native UX, real-time systems, and AI services at once.
What to Watch Next
Several signals are worth tracking in this repository.
- Release cadence: GitHub releases show active desktop updates in April 2026.
- Plugin growth: Omi’s official help center positions plugins/apps as a key extension surface.
- Local-vs-cloud balance: The project’s long-term appeal may depend on how much control users get over storage and processing paths.
- Hardware integration maturity: Omi’s differentiation rises if wearable and glasses-based flows become easier to deploy and govern.
Conclusion
Omi is an open-source AI memory platform by BasedHardware for capturing conversations and screen activity, turning them into transcripts, summaries, actions, and queryable memory across devices. That identity is the key to reading the repository correctly.
This is not just a desktop note-taker, and it is not just a hardware experiment. It is a multi-layer product stack that combines apps, firmware, backend services, and extensions around one central bet: that personal AI becomes more useful when it can remember continuously and act on that memory later.
Whether that vision proves durable will depend on privacy design, workflow usefulness, and ecosystem adoption. But as a repository, Omi already stands out as a serious, open implementation of the “AI second brain” idea.