Overview
Cloud notetakers upload your microphone to someone else’s server. Notova inverts that. Audio is captured, transcribed and summarized on the device, and the backend is scoped so tightly that it cannot receive audio at all.
That constraint drives the design. If the server cannot do the AI work, the client has to, on three platforms, without writing the pipeline three times.
Architecture
- One shared core, three shells. Domain models, the
PipelineService, and the on-deviceTranscriber/Summarizerseams live in local Swift packages. The iOS (SwiftUI) and macOS (NavigationSplitView) apps both depend on that core and differ only in platform plumbing, such asMacAudioRecorderwrapping AVAudioEngine for capture and file import. - Android is a parallel implementation in Kotlin and Jetpack Compose that matches the same record → transcribe → summarize contract.
- Model seams, not model calls. Transcription and summarization sit behind protocols, so a real engine drops in without touching any call site. Today they are stub implementations; the intended engines are Whisper for transcription and Gemma 3n E4B for summarization.
- Generated project files. XcodeGen produces the Xcode project from
project.ymlinstead of committing it, so build configuration is reviewable as source and binary project-file merge conflicts go away. - Swift 6 with strict concurrency across the app, targeting iOS 17 and up.
A backend defined by what it refuses to do
- Accounts with JWT access and refresh tokens. Passwords are hashed with argon2id, and refresh tokens are stored only as SHA-256 hashes, so they can be revoked without the server ever holding the secret.
- An OAuth2 token broker for Google, Notion, Slack and Salesforce, using Authorization Code with PKCE. Third-party tokens are encrypted at rest with AES-256-GCM.
- Metadata-only sync. Title, duration, source, status and timestamps sync across devices. No audio, transcripts or summaries are stored at rest.
- Pass-through export. A note bound for a connected service goes device → backend → third party. It is forwarded, never persisted.
- Built on Node 22+ and TypeScript ESM with Hono, Drizzle ORM over better-sqlite3, zod for validation, jose for JWT, and Vitest.
What this design buys
- A privacy guarantee that is structural, not a promise: there is no code path by which audio reaches the server.
- One behavioral contract to test instead of three, because the shared core owns the pipeline and the shells own only capture and presentation.
- Model upgrades that cost nothing at the call sites, since the engines sit behind protocol seams.
Current status
The iOS, macOS and Android shells, the shared core, and the backend work. The transcription and summarization engines are still stubs: the seams are real and tested, but the models are not yet dropped in. Meeting Bot, below, is where the on-device Whisper pipeline runs today.