Overview
Most assistants are a thin client over someone else’s inference. Anu inverts that: the model lives on the phone, the cloud is opt-in, and every message is classified locally before a single token is generated.
I built it in a two-week sprint for the Apple Swift Student Challenge, and it won. The hard constraint was not the deadline but the privacy posture. Once memory never leaves the device, a lot of convenient architecture is off the table.
What I built
- On-device model. Gemma 4 E4B runs through Google’s LiteRT engine from a
.litertlmbundle, loads in roughly a second, and reuses the KV cache across turns. Core ML Gemma 3 4B is the secondary fallback. - Two classifiers before generation. A task classifier and a model classifier both run on Apple’s
NLEmbeddingsentence embeddings. The task classifier scores 86.7% on a 60-case eval. Both are confidence-gated: a low-confidence message falls through to the agent loop instead of being forced into a bucket. - ReAct agent loop over calculator, web search and a Gemini-escalation tool, with structured
{"tool_call": …}parsing and a reasoning trace the user can read. - Voice first.
SFSpeechRecognizerhandles dictation on-device; the microphone is the primary control and auto-sends on stop. - Editable memory. Saved notes are sources, not hidden state. They are visible and editable in Settings, retrieved by embedding similarity, added to the prompt as numbered
[M#]ground truth, and cited back in the trace. - Multimodal input. Attached images go straight into E4B’s native vision encoder.
Privacy as an architectural property
- API keys live in the Keychain, and speech recognition is pinned to on-device.
- A two-layer PII scrub runs before any cloud call is permitted.
- Memory never leaves the device. There is no sync path for it.
Built like a product, not a demo
Nothing in the product is mocked: a missing model surfaces as an explicit error, not a silent fallback to canned text. Tool calls have timeouts, in-flight work is cancellable, conversations persist, and Siri and Shortcuts integration runs through App Intents. CI keeps 300+ unit and UI tests green, and the app builds and runs on iOS 17 through 26.5.
Limits, by design
A sandboxed third-party app cannot be the system assistant. The “Hey Siri” default, on-screen awareness across other apps, Safari and system control, and OS-level personal-context indexing are Apple-only surfaces. They are documented as out of scope rather than stubbed to look complete. Where Anu goes further than the system assistant is the part it controls: fully on-device by default, with a swappable model.