Skip to main content
Three models can fill the TTS slot. They differ in backbone, in how a voice is chosen, and — the part that matters most to a client — in what they put on the wire. This page covers all three and the format negotiation that lets one client decode any of them.

​
Available models

Set the one you want with TTS_MODEL in model-server/.env. Statuses are from model-server/models.yaml.
Only indic-parler has been run on VoicEra’s hardware. models.yaml records “Not yet run on hardware” against both orpheus and indic-mio, and their folder READMEs repeat it: for indic-mio, “neither container has been built or started”. ready means the folder exists with a Dockerfile, not that the model is verified here.
omnivoice is planned and needs its own runtime — it is diffusion-style, so not vLLM-servable.

​
indic-parler

AI4Bharat’s Indic Parler TTS with a custom inference engine: paged KV cache, continuous batching, CUDA graphs, and incremental DAC decoding. Covers the same 23 Indic languages as indic-conformer. The voice is chosen by a free-text description rather than a fixed preset list, so voice and instructions are recomposed into one prompt on the server. tests/test_tts_request_parity.py pins that they recompose into exactly the prompt the model used to get.
fetch.sh in the folder downloads the checkpoints into checkpoints/. The response carries X-Sample-Rate (44100) and X-Audio-Format. Chunked HTTP can split a 4-byte float across reads, so a client that concatenates chunks blindly gets noise — carry the remainder forward. tests/test_pcm_chunk_boundaries.py pins that. It needs an NVIDIA GPU of Ada generation or newer; the engine leans on CUDA graphs and flashinfer. Warm time-to-first-audio on an H200 is around 250 ms, roughly 1.5 s cold. The Parler checkpoint comes from Drive, but the tokenizer and T5 encoder are pulled from ai4bharat/indic-parler-tts, which is gated. See Running on GPUs for the two ways round that.

​
orpheus

A Llama-3.2-3B backbone with a SNAC codec, served by vLLM with continuous batching. 23 Indian languages on the current (v2) roster. The speaker name picks the language. Every speaker in the roster belongs to exactly one language, so voice="Amit" is Hindi and voice="Anitha" is Tamil. voice and language are therefore not independent — which is different from indic-parler, where the voice is a free-text description and language is a separate field. src/orpheus_server/ is the upstream project as its authors wrote it, excluded from VoicEra’s ruff config on purpose. Two changes were made, both about fitting the slot rather than changing the model:
  1. Port. The image listened on 9000; the TTS slot is addressed as tts:8002. PORT is honoured so the folder is not welded to VoicEra’s numbering.
  2. Self-description. POST /v1/audio/speech now sets X-Audio-Format, X-Sample-Rate and X-Channels. It already sent X-Language and X-Voice.
There is no fetch.sh — vLLM downloads the weights from HuggingFace into the hf_cache volume on first start. First start therefore takes several minutes with nothing on /health; watch docker compose logs -f tts rather than assuming it has hung. /health returns 503 while loading and 200 once warmup and CUDA graph capture have finished, which is exactly what the gateway’s probe wants. The upstream server also exposes /v1/tts, /v1/tts/stream, a /v1/tts/ws WebSocket, /v1/voices, /v1/styles and /metrics. The gateway forwards only the OpenAI routes, so those are reachable with docker compose exec for debugging but are not part of the slot contract. Worth knowing if you touch the streaming path: the authors measured which formats survive being sent in chunks, and flac does not — libsndfile seeks back and patches the header at close, and that patch never reaches a client whose first bytes already went out. Only pcm and mp3 stream. Roughly 7 GB for a 3B backbone in bf16 plus cache. Being vLLM-backed, it takes a memory reservation at startup the same way the LLM slot does; the KV-cache ratios are documented in tts/orpheus/config.yaml.

​
indic-mio

A Qwen3-0.6B backbone with the MioCodec vocoder. Voice is a preset speaker embedding rather than a text description, and the embedding carries the accent, so language is informational here. Two containers, and why. This is the first model here that is not one process. Token generation is delegated to a vLLM sidecar serving SPRINGLab/Indic-Mio; the model’s own container does only the MioCodec decode and the orchestration. That is how upstream runs it, and it is not worth undoing: the model’s Dockerfile pins torch 2.7.1/cu128 for Blackwell sm_120 kernels, and vLLM’s own image brings a different torch. So the folder brings the sidecar with it, in compose.extra.yml. The slot contract is unchanged — one service named tts on 8002 answering the OpenAI route — and the sidecar is an internal service only tts talks to, publishing nothing on the host. scripts/start-model-server.sh adds the overlay automatically when this model is selected, by looking for the file rather than knowing the model.
Watch the sidecar’s memory setting. Upstream uses --gpu-memory-utilization 0.35, which is a fraction of the card’s total memory — on a 143 GB H200 that reserves about 50 GB, on a GPU shared with production through MPS, which does not partition memory. The overlay defaults to 0.06 (~8.6 GB) and exposes MIO_VLLM_GPU_MEMORY_UTILIZATION to change it. This is a second reservation on the same card as VLLM_GPU_MEMORY_UTILIZATION, not an alternative to it.
The upstream server spoke a WebSocket protocol — one JSON message in, float32 frames out, {"type":"done"} to finish. That is gone. server.py now serves: The engine is untouched; only the transport changed, and it bought two things. Cancellation is free — synthesize_stream is an async generator, so when the caller hangs up mid-sentence Starlette closes it, GeneratorExit unwinds into the aiohttp response reading from vLLM, and that request is aborted. Self-description — responses carry X-Audio-Format, X-Sample-Rate and X-Channels. UPSTREAM-README.md in the folder is the original documentation, kept verbatim; note that it still describes the WebSocket protocol. No fetch.sh: the codec’s weights come from HuggingFace on first start into the hf_cache volume, and the backbone comes down in the sidecar. Watch docker compose logs -f tts vllm-mio rather than waiting on /health, which stays 503 until the codec has loaded.

​
Format negotiation

Two TTS models in this slot disagree on the wire, and neither is wrong: OpenAI’s response_format vocabulary is mp3, opus, aac, flac, wav, pcm — so Orpheus is the compliant one, and pcm_f32le is an extension Indic Parler serves because float32 is what its engine produces. The client therefore cannot assume a width or a rate. It reads X-Audio-Format and X-Sample-Rate off the response and decodes accordingly, which is why the contract requires a model to declare them. Getting this wrong does not raise an error — it produces plausible bytes that sound like noise on a phone line. A format the client cannot decode produces a clear error naming it, never silence. tests/test_tts_format_negotiation.py pins both the decoder table and the chunk-boundary handling, which is width-dependent: a sample split across two HTTP reads desynchronises everything after it, and a 2-byte model re-opens that bug if the width is hardcoded to 4. The test extracts the decoder from the real client source, so it fails when that drifts rather than passing against a copy. It also checks that gain never wraps.

​
Barge-in and GPU release

TTS moved off WebSockets to plain HTTP, which gave cancellation for free. Direction of travel decides the transport: TTS is one-directional — text in, audio out — so HTTP does the same job with less machinery. When the caller interrupts, Pipecat stops reading the response, the connection drops, and that drop travels through the gateway to the TTS server, which frees the GPU slot. For indic-parler the server evicts the request from the batch; for indic-mio the async generator’s GeneratorExit aborts the vLLM request. Either way generation stops rather than finishing a sentence nobody is listening to. That propagation is what tests/test_gateway_streaming.py checks: the gateway streams rather than buffers, and a client disconnect reaches the upstream. The stopping-work-on-hangup row of the container contract exists for exactly this.

​
Voices

indic-parler — free-text description. models.yaml records one preset, Divya, plus free_text_description: true and no cloning. orpheus — a fixed roster, currently tts/orpheus/voices-v2.json: 23 languages, with 14 speaking styles (news is the default — AIR style news, TV style news, educational lecture, single person narration audiobook, children's stories, advertisements, Customer Care, and six emotion tags: happy, sad, anger, fear, surprise, disgust). Every speaker name is unique, which is what lets an OpenAI client pick a language using only the standard voice field. GET /v1/voices lists them.
The folder also still carries the older voices.json (v1): 12 ALL-CAPS styles (CONV default, plus WIKI, NEWS, BOOK, …) across the same 22 languages minus Bhili, and a completely different style vocabulary from v2 — the two rosters are not interchangeable, model-server/tests/test_orpheus_roster.py pins that compose.extra.yml’s ORPHEUS_VOICES_FILE matches whichever checkpoint models.yaml records for this slot. Which file is actually loaded is a deployment detail, not a docs one — check ORPHEUS_VOICES_FILE in the running compose.extra.yml before trusting either list. VoicEra’s indic_orpheus provider (see Providers → Indic Orpheus) is written against v2.
indic-mio — five preset speakers built from AI4Bharat Rasa reference clips: Aditi (default), Meera, Ananya, Rahul, Arjun. It is a zero-shot voice-cloning model, so a voice is a speaker embedding derived from one reference clip. The embedding is timbre only, so one voice works across all 22 Indic languages plus English — you do not need a voice per language. The layout is a manifest plus one clip each:
At startup the server derives each voice’s embedding from its ref clip once and caches it under MIO_VOICES_CACHE_DIR (default inside the HuggingFace cache volume, so it persists). If manifest.json is absent or no ref clip is usable, the server falls back to a single legacy embedding rather than failing. To add a voice, add an entry {name, gender, ref} to manifest.json and a matching mono wav in refs/. name is the voice id shown to users and stored in the agent’s tts_model.speaker, so keep it stable once shipped. scripts/build_voices.py pre-bakes the embeddings so first boot is instant; scripts/fetch_rasa_refs.py curates new candidates from the gated ai4bharat/Rasa dataset (CC-BY-4.0 — see voices/NOTICE).