REFERENCE PATTERN / ILLUSTRATIVE
- Microphone
- Local ASR runtime
- Transcript
- Compose UI
Voice features are easy to describe and hard to ship well. A button captures speech, a model returns text, and a user can continue. Between those moments lie microphone permissions, audio buffering, model packaging, lifecycle handling, network assumptions and the question of where sensitive audio is processed.
The Dayu Android project includes on-device speech recognition with sherpa-onnx and ONNX Runtime, within a native Kotlin and Jetpack Compose application. This note explains the architecture and trade-offs involved in that kind of integration. It does not imply that every AI or communication feature in Dayu runs offline.
Why put recognition on the device?
Local inference changes the operating assumptions. The recognition path no longer requires raw audio to be sent to a speech-recognition server for every request. Depending on implementation, that can make certain features usable with limited connectivity and can reduce the transfer of microphone data to external speech services.
But on-device does not mean cost-free, perfectly private or permanently offline. A device still needs to load model files, allocate memory and spend CPU or accelerator time. Other application features may continue to use backend APIs, network services or server-side model processing.
The processing boundary
It helps to separate the feature into five responsibilities: user intent, audio capture, local inference, result interpretation, and UI presentation.
A Compose screen should describe states such as idle, requesting permission, listening, processing, result and failure. The Android audio layer handles samples and resource ownership. A recognition adapter wraps the inference runtime. The rest of the application consumes normalized transcript events rather than implementation-specific decoder objects.
That separation is valuable during change: a different audio format, model build or recognition strategy should not require rewriting every screen.
Model distribution is a product decision
An on-device speech model must reach the handset somehow. Bundling it with the install simplifies first use but grows the package. Downloading it on demand reduces initial footprint but adds connectivity, integrity, storage and lifecycle requirements. Both choices are defensible; the right one depends on model size, device support and when recognition must become available.
Questions to answer include: Which languages and accents are in scope? How much storage can the feature use? What should happen on a lower-memory phone? How are model versions updated? What is the recovery path if a download or load fails?
The Dayu case documents sherpa-onnx and ONNX Runtime as the local ASR foundation. It does not publish a verified performance benchmark or establish a universal device compatibility claim.
Streaming and lifecycle behavior
Audio rarely arrives as one perfectly formed block. A production recognition interface must decide how to handle partial hypotheses, silence, cancellations and interruptions. If the user leaves the screen, locks the device or receives another audio-related event, recognition should not continue using the microphone indefinitely.
One useful architecture treats the recognizer as an explicit state machine. The UI observes states and events; a lifecycle-aware coordinator owns the capture session; the inference adapter returns partial and final text. A cancel action must release audio resources, not only hide the animation.
A state sequence to reason about
Idle → permission check → Listening → Partial transcript
↘ denied → recoverable message
Listening → Final transcript → Confirmation / edit
Listening → Cancelled → release microphone → Idle
Listening → Error → release resources → Retry or exit
This is an illustrative engineering model, not a diagram copied from Dayu's internal implementation.
Privacy claims need narrow wording
"Speech recognition runs on-device" can be accurate for one feature without implying "the entire app never transfers personal data." For example, text produced locally may later be submitted to a separate messaging or AI feature at the user's request. The user-facing disclosure needs to describe that second action independently.
Engineering tests should verify the recognizer's network dependencies rather than infer them from marketing terms. Microphone permissions, crash reports, analytics hooks and retention of temporary audio files also deserve review.
For Dayu specifically, the existing project notes distinguish local ASR from a separate server-side prompt-based assistant. The team's LoRA fine-tuning experience is genuine model-development work, but it is not presented as the inference path deployed in the Dayu product.
Practical test matrix
A credible release test spans more than a quiet-room demo. Try first-run permission denial and later approval; repeated start/stop; background interruption; low available storage; model loading failures; noisy audio; and several device classes. Verify that the UI does not promise a transcript after cancellation and that the microphone indicator disappears when the session ends.
Measure latency, power and recognition quality on your actual target devices before making public numerical claims. Model benchmarks from another device or language cannot substitute for that evidence.
What the trade-off really is
Cloud speech APIs can simplify distribution and move compute cost to a remote service, but introduce network and data-processing dependencies. Local ASR shifts more responsibility to mobile engineering: model delivery, compatibility, thermal behavior and lifecycle correctness.
Neither approach is universally better. The right question is which boundaries the feature needs — for connectivity, privacy, performance and ongoing maintenance — and which implementation can satisfy those boundaries with verifiable behavior.
Related engineering work: Dayu Android case study. If you're designing speech-enabled mobile software, discuss the requirements.