Voice: STT, TTS, latency, and quality tuning

The voice stack is what makes a phone call feel real. This page covers the engines, the trade-offs, and how to tune for your use case.

The pipeline

Three primitives running in parallel:

1. STT — streaming speech-to-text, transcribes as the caller talks.
2. LLM — reads the transcript, generates a response.
3. TTS — synthesizes the response audio and streams it back.

End-to-end target: under 500ms from "caller stops talking" to "agent starts speaking."

STT: Deepgram Nova-3

Deepgram's Nova family — streaming, ~150ms transcript latency. If the agent has a fixed language, STT is pinned to it (more accurate). If the agent language is Multilingual, Deepgram detects the language per turn.

STT is not a per-agent setting. Whisper exists only as an internal fallback if Deepgram is unreachable.

Voice modes

Pick per agent on AgentDetail → Voice. Preview every voice before committing. An earlier version of this page listed Cartesia and OpenAI TTS as selectable engines. OpenAI TTS is not used; Cartesia is only an automatic fallback when ElevenLabs is unreachable (corrected 6 October 2026).

Languages

The agent language picker offers Turkish, American English, British English, German, French, Spanish and Arabic, plus Multilingual. The ElevenLabs voices are multilingual, so the same voice speaks every language; for Turkish, the Turkish-native voices (e.g. Selin) sound most natural.

Latency budget

Where the 500ms goes:

• STT: ~150ms (per chunk)
• LLM first-token: ~150–300ms (depends on model)
• TTS first-audio: ~80–150ms
• Network: ~30–50ms

To keep under 500ms median:

• Use GPT 5.4 Nano or GPT 5.4 Mini for the LLM (faster than the larger siblings).
• Stay on Standard mode — ElevenLabs Flash v2.5 is already the low-latency TTS.
• Keep system prompt under 800 words (longer prompts → slower first-token).
• Don't over-retrieve from KB (top-3 is usually enough).

For deeper detail see the sub-500ms latency guide.

Background sound

You can layer ambient audio under the agent's voice — typing, café murmur, office hum. Lives on AgentDetail → Speech Settings → Background sound. Useful for:

• Making AI calls feel less sterile.
• Matching the expected acoustic environment (office, restaurant, retail).
• Masking subtle audio glitches.

Pick from the gallery or upload your own short loop.

Voice consistency across languages

If your agent runs in multiple languages on the same call (rare but happens), pick a multilingual voice so the character stays consistent. Switching voices mid-call is jarring; a multilingual ElevenLabs voice keeps the same character in every language.

DTMF (keypad) input

Voice agents accept keypad presses. Use cases:

• Language menu: "Press 1 for English, 2 for Turkish."
• Account number entry.
• IVR-style routing on top of the voice agent.

Configure the menu in the system prompt; the runtime parses DTMF tones and feeds them to the LLM as transcript text.

Voicemail detection

Agents detect voicemail (the "leave a message after the beep" pattern) and respond per your config:

• Hang up — don't leave anything.
• Leave a scripted message — define what to say.
• Treat as human — risky; only with very short scripts.

Detection is on by default. Tunable per agent.

Recording quality

Call recordings are MP3 at 16kHz mono — phone-quality. Storage in our buckets, retention configurable per workspace.

If you need higher-quality audio (e.g. for legal review), turn on raw WAV recording — it costs more storage but preserves full fidelity.