Local speech models

Listen and speak on the computer running Sleeptalk, with no speech account or API key. Local is a Cascade speech option: your configured agent still handles the conversation and can use background workers.

Models and downloads

The default model downloads total about 493 MB, plus the optional speech runtimes and space for unpacking. These models are downloaded during setup; they are not bundled in the base Sleeptalk wheel. Download size is not a RAM requirement. Existing profiles keep explicitly selected models and voices.

Moonshine Small streams revisable partial text while you speak. Moonshine Tiny transcribes a completed utterance. Streaming runtime wheels cover Apple silicon on macOS 15+ and Linux x86-64/ARM64; Intel Macs can use Tiny.

Set it up

1. Open sleeptalk or sleeptalk settings and choose Local under Speech.
2. Install the optional local runtimes if prompted. Streaming listening uses sleeptalk[local,local-streaming].
3. Download the selected models. Sleeptalk checks each download against a pinned SHA-256 before use.
4. Run the bundled transcription and voice checks, then make a real call to check your microphone and the spoken reply.

No speech-key prompt appears. Headless setup can download the models but does not silently install missing runtime packages; follow its install hint.

[stt]
provider = "local"
model = "moonshine-small-streaming-en"

[tts]
provider = "local"
model = "kokoro-en"
voice_id = "af_heart"

Keep the profile's agent, workspace, policy, and destination. Editing TOML does not install runtimes or download models.

Voices and custom reference recordings

Kokoro offers 28 English voices, with Heart as the default. Piper's Lessac is the smaller download. Choose a voice in settings; Sleeptalk selects its model and downloads it when needed. Older profiles that use Lessac keep it.

Kyutai Pocket TTS offers Bria and Loona, plus Create a Kyutai voice in Local speech settings. Upload a clear recording of one speaker, ideally about 10 seconds, within the 3–30 second and 20 MB limits. WAV, MP3, and M4A work when your browser supports them. Name it, save it to the library, preview it, then choose Use voice and save the speech choices. A successful upload does not guarantee a close speaker match.

Reference recordings stay on the Sleeptalk host under ~/.sleeptalk/voices/pocket, or $SLEEPTALK_HOME/voices/pocket. Back up this directory if you move the host. No external speech API or training run is needed.

Pocket's pinned ONNX archive contains CC-BY-4.0 metadata and a README that also says non-commercial. Check that bundle's terms before choosing it for your use; do not substitute the upstream repository's license for the shipped archive's metadata. The repository guide links the exact bundle and explains the reference-encoder fix.

Responsiveness and background work

Local browser calls use early turn scoring, faster barge-in, and short opening phrases. The next phrase can be synthesized while the current one plays. Piper and Kokoro still synthesize each phrase as a batch; this is not continuous generation within a phrase. Interrupting discards pending audio; native compute can finish its current step before another synthesis starts.

Agent generation time, endpointing, model loading, and competing CPU work still affect when you hear a reply. Component benchmarks are not a promise of total call latency. Several workers can run with Local speech, but their builds or tests share the host's CPU; lower policy.max_background_jobs if speech stutters.

Cache and offline preparation

Download the speech models and your local language model while online. Use a same-machine call and test it with the network disconnected before relying on an offline setup. Remote browser delivery and Discord still require a network.

Models live under ~/.cache/sleeptalk/models, $XDG_CACHE_HOME/sleeptalk/models, or SLEEPTALK_MODELS_DIR. Kokoro v1.0 uses a separate kokoro-en-v1_0 cache folder, so an older v0.19 download does not satisfy the new model. Choose Local in settings after upgrading to fetch v1.0. Old files are left intact.

Local listening and a cloud voice, or the reverse, can also be combined through Mix providers. See voice modes and speech providers.