The brief for the Voice-First Learning Assistant was clear: build a system that delivers personalised, voice-driven educational content to visually impaired students in Nigeria. What wasn't in the brief — because nobody thinks to write it down — is that Nigerian classrooms have unreliable power, intermittent internet, and ambient noise that would defeat most commercial speech recognition systems.
The Hybrid Architecture Decision
Early prototypes used the Google Cloud Speech-to-Text API exclusively. They worked fine in demo conditions. They failed completely during a school visit where the wifi cut out for 40 minutes. We needed a fallback.
The final system uses a two-tier STT architecture:
- Primary: Google Cloud Speech-to-Text (en-NG locale) — highest accuracy when online
- Fallback: Faster-Whisper running locally on-device — slower but network-independent
The client detects connectivity and routes automatically. Accuracy dropped from 97% to 91% in offline mode. For a student whose alternative is no access at all, that's acceptable.
The Nigerian Accent Problem
Generic English STT models are trained on American and British speech. Nigerian English has distinct prosody, vowel shifts, and code-switching patterns. The first test with a standard model produced transcription errors on approximately 30% of student utterances.
Selecting Google's en-NG locale was the single biggest accuracy improvement — jumping from 70% to 94% accuracy on our test corpus of student speech. The Faster-Whisper model was fine-tuned on a small dataset of Nigerian-accented educational phrases to close the remaining gap.
Non-Blocking TTS: The Listening Loop Problem
Early versions had a subtle but severe bug: the TTS engine spoke to the student, and the microphone picked up the system's own voice as student input. The VAD (Voice Activity Detection) would then try to transcribe what the system just said — and respond to itself.
The fix was implementing non-blocking TTS via Python background threading. The microphone is gated during TTS playback and reopened when the audio stream completes. Combined with a configurable VAD sensitivity tuner, false triggers dropped to under 2% in field tests.
Lessons for Offline-First AI Builders
- Design for your worst connectivity scenario first, not your best
- Use locale-specific models — generic models are built for someone else's users
- Background thread your audio I/O or you will create infinite loops
- YAML-based configuration for VAD parameters — teachers need to tune without redeploying
The system is still in active development. The next phase is adaptive content difficulty using Bayesian student modelling. But the core offline-first architecture is stable. It works in rooms with no signal, 40-degree heat, and a classroom full of talking children. That's the real benchmark.