Providers (apps/providers)

The providers package — layout, public API, and dependency model.

apps/providers holds Pydantic configs and a factory that build Pipecat STT, TTS, and LLM services. It is a library, not a container: it is copied into both the api and runtime images. The API imports it to generate provider catalogs; the runtime imports it to build live services for a call.

This page walks through the package layout and public API. Why the registry is shaped this way, and what it buys you, is in Provider registry. To add a vendor, follow Adding an AI provider.

Public API

from apps.providers import (
    AgentConfig,
    Kind,
    create_stt_service,
    create_tts_service,
    create_llm_service,
)
from apps.providers.languages import LANGUAGES, label

agent = AgentConfig.model_validate({
    "stt_config": {"provider": "deepgram", "api_key": "...", "model": "nova-3"},
    "tts_config": {"provider": "openai", "api_key": "..."},
    "llm_config": {"provider": "openai", "api_key": "...", "model": "gpt-4.1-mini"},
})

stt = create_stt_service(agent)
tts = create_tts_service(agent)
llm = create_llm_service(agent)

cloud/factory.py is a compatibility re-export of the same symbols and is marked deprecated. New code imports from apps.providers or apps.providers.factory.

Schema dump for the API

Provider availability, field requirements, secrets, suggested models, and supported languages live on the config classes. Dump a readable catalog — not raw JSON Schema — rather than maintaining a second list in a router or a UI:

from apps.providers import Kind, provider_schemas, configuration_defaults

openai = provider_schemas(Kind.LLM)["openai"]
# {
#   "provider": "openai",               # from provider: Literal["openai"] = "openai"
#   "name": "OpenAI",                   # UI display name from config.name
#   "provider_type": "cloud",           # cloud | adapter | local
#   "description": "...",
#   "required": ["api_key"],
#   "secrets": ["api_key"],             # auth fields at a glance
#   "fields": {
#     "api_key": {"type": "...", "secret": true},
#     "model": {"type": "string", "examples": [...], "input_mode": "both"},
#     "base_url": {"type": "string", "input_mode": "input"},
#   },
# }

deepgram_lang = provider_schemas(Kind.STT)["deepgram"]["fields"]["language"]
# deepgram_lang["examples"]         → flat canonical ids
# deepgram_lang["model_options"]    → model → [canonical ids]
# deepgram_lang["language_codes"]   → model → {canonical: vendor_code}
# deepgram_lang["input_mode"]       → "options" | "input" | "both"

defaults = configuration_defaults()

Catalogs are derived from the discriminated unions in factory.py, so there is no $defs, $ref, or anyOf to walk. provider_type comes from the config module path. input_mode is derived from Field examples and model_options plus allow_custom_input: options for a list only, input for free text, both for select-or-type. Secrets carry no input_mode. Language extras come from each vendor’s STT_CAPABILITIES / TTS_CAPABILITIES via languages_map() and language_schema_extra().

These dumps are what GET /api/v1/configuration/{stt,tts,llm} returns.

Layout

Path	Role
base.py	Shared Kind / ProviderType enums and Auth/Settings/Config bases
registry.py	@register_* maps, load_providers(), creator helpers (api_key, llm_settings)
factory.py	Discriminated unions from registry + thin create_* dispatch
schema.py	provider_schemas / configuration_defaults readable catalog dump
languages.py	Canonical language ids → labels; language_schema_extra()
cloud/<vendor>/	Pipecat-backed vendors (catalog, config, languages, service) → provider_type=cloud
adapters/<vendor>/	First-party services (service.py + optional tts.py) → provider_type=adapter
local/<vendor>/	VoicEra’s own model server, reached via the gateway rather than a vendor SDK → provider_type=local

Each vendor config.py stacks three layers. Auth holds credentials, with secrets marked json_schema_extra={"secret": True}. Settings holds vendor knobs — voice, speed, base_url, grpc_url. Config combines both with the Base*Config fields provider, model, and language. Credentials never live on the bases, and endpoints and hosts belong on Settings, not Auth.

Provider types

The inventory is 22 cloud vendors, 2 adapters, and 2 local providers. The adapters are bhashini (STT and TTS) and kenpath (LLM only); the local providers are indic_orpheus (TTS) and indic_nemotron (STT) — so no single kind lists all 26. This table was generated by calling provider_schemas() on the checked-out tree:

from apps.providers import Kind, provider_schemas
for kind in (Kind.STT, Kind.TTS, Kind.LLM):
    for pid, entry in sorted(provider_schemas(kind).items()):
        print(kind.value, pid, entry["name"], entry["provider_type"])

Kind	Count	Providers
STT	13	assemblyai, azure_speech, bhashini (adapter), cartesia, deepgram, elevenlabs, gladia, google, indic_nemotron (local), openai, sarvam, smallest, speechmatics
TTS	15	azure_speech, bhashini (adapter), camb, cartesia, deepgram, elevenlabs, google, indic_orpheus (local), inworld, lmnt, openai, rime, sarvam, smallest, xai
LLM	10	atlascloud, aws_bedrock, azure_openai, google, google_vertex, groq, kenpath (adapter), openai, openrouter, sarvam

Many vendors appear under more than one kind — openai, google, sarvam, deepgram, cartesia, elevenlabs, azure_speech, and smallest each register two or three. Credentials are stored once per vendor in ProviderAuth and shared across its slots. indic_orpheus and indic_nemotron have no stored credentials at all — see The local providers.

Regenerate this table rather than editing it by hand. provider_schemas() reads the registry, so it cannot drift from the code the way a hand-written list does.

Dependencies and extras

Core: pydantic, loguru, pipecat-ai plus the per-vendor Pipecat extras. The Bhashini adapter needs grpcio, protobuf, and numpy for TTS; websockets for Dhruva WebSocket STT; python-socketio[asyncio_client] for English Socket.IO STT; and tritonclient[grpc] for Bhili NVCF STT. The Kenpath adapter needs PyJWT and cryptography, both in apps/runtime/requirements.txt.

The two images install only what they use. apps/runtime/requirements.txt pins pipecat-ai[deepgram,cartesia,openai,silero,websocket]==1.8.1 — five extras, not twenty-two. apps/api/requirements.txt does not install pipecat-ai at all.

That’s deliberate, which is why the next section matters.

Import-time safety

Every vendor’s service.py imports Pipecat inside its creator function, never at module scope:

@register_tts
def create_tts(cfg: BhashiniTTSConfig):
    from .tts import BhashiniTTSService   # imported on call, not on import

    return BhashiniTTSService(...)

load_providers() imports every *.service module under cloud/, adapters/, and local/ so the @register_* decorators run. If a vendor imported its Pipecat extra at module scope, that discovery sweep would raise ImportError on the first vendor whose extra is not installed — and the whole catalog would be unavailable.

Deferring the import means the registration decorator runs with no vendor SDK present. The API can therefore enumerate all 26 providers, validate agent configs against them, and serve GET /configuration/* while installing none of them. Only the runtime, and only at the moment it creates a service for a call, needs the extra on disk.

apps/providers/__init__.py extends the same idea to the package root. AgentConfig, STTConfig, TTSConfig, LLMConfig, and the three create_*_service functions are resolved lazily through a module-level __getattr__, because factory.py imports loguru. Importing apps.providers for a catalog dump therefore does not pull in the factory at all.

A provider that appears in a catalog is not proof its SDK is installed. create_*_service raises at call time — during a live call — if the extra is missing. When you enable a new vendor, add its extra to apps/runtime/requirements.txt and rebuild the runtime image before you route a call to it.

The adapters

Bhashini

adapters/bhashini/ holds first-party Pipecat service subclasses rather than wrappers over a Pipecat vendor integration. STT has three backends selected by catalog model in service.py: stt.py (Dhruva WebSocket Indic Conformer), socketio_stt.py (Dhruva Socket.IO Whisper for English), and bhili_stt.py (NVCF Triton gRPC for Bhili). TTS is tts.py (NVCF gRPC). service.py registers both kinds with @register_stt and @register_tts exactly like a cloud vendor.

Bhashini covers STT and TTS. Dhruva STT backends authenticate with api_key. Bhili STT uses its own NVCF pair bhili_auth_token + bhili_function_id (distinct from TTS even when the token value matches). TTS uses auth_token + function_id. Bhili’s catalog language is canonical bh (see languages.py); the Triton STT wire code is bhb and the NVCF TTS wire code is bhli (Marathi speakers), both applied at service construction via resolve_wire_language.

One Bhashini ProviderAuth holds all five secrets — provider_level_auth("bhashini") reports api_key, bhili_auth_token, bhili_function_id, auth_token, and function_id. Keep Bhili STT and TTS function ids separate; they are different NVCF functions.

Kenpath

adapters/kenpath/ is an LLM adapter for Maharashtra Vistaar and Bharat Vistaar. It registers with @register_llm and is LLM only.

Auth	Three RSA PEM secrets matching mono kenpath_llm: private_key (jwt_private_key.pem, Vistaar iss: voice-provider), bharat_prod_private_key (prod_private_key_bh.pem), bharat_dev_private_key (dev_private_key_bh.pem) for Bharat iss: samvaad. Catalog MODEL_AUTH_SECRETS / resolve_auth_secret(model) selects which field is required.
Model as environment	model selects backend + environment: vistaar-prod (Marathi, Bhili), vistaar-dev (Marathi, Bhili), bharatvistaar-prod (English, Hindi), or bharatvistaar-dev (Indic). Each maps to its own base URL (and Bharat completions path) in catalog.py; base_url overrides the host.

Vistaar: source_lang / target_lang default to mr (mr, bhb). Marathi uses streaming GET {base_url}/api/voice/ (JWT always). When source_lang is bhb, Kenpath calls the catalogued Voice Bhili URL (…/api/voice-bhili/) with JSON response. Voice Bhili follows redirects and sends JWT only on Vistaar prod.

Bharat Vistaar: implemented in bharat_vistaar_llm.py (dispatched from service.py). No Voice Bhili. Uses POST {base}{/api/v1/chat/completions|/api/v1/chat-dev/completions} with OpenAI-style SSE (model: bharatvistaar-voice), headers X-Tenant-ID / X-User-ID / X-Session-ID / X-Language (source_lang), and full chat history. Prod languages: en, hi. Dev adds Indic codes (bn, te, mr, ta, gu, kn, ml, as).

Kenpath is the one provider that reads the call id. run_pipeline() calls llm.set_call_id(call_id) when the service exposes that method — Vistaar session_id / Bharat session headers. Without it each turn falls back to a fresh uuid4().

private_key, bharat_prod_private_key, and bharat_dev_private_key are secrets stored encrypted in ProviderAuth. They are multi-line PEMs — send with real newlines in the JSON string, not escaped literals. Do not mix Vistaar and Bharat keys (wrong PEM → 401 signature verification failed).

Both adapters

The registry treats them identically to any cloud vendor at every layer — discovery, discriminated union, catalog dump, dispatch. The only difference a caller sees is provider_type: "adapter" in the schema, which exists so a UI can group or label it.

The local providers

local/ holds providers that talk to VoicEra’s own model server gateway instead of a third-party API. Two ship today.

Indic Orpheus (TTS)

local/indic_orpheus/ wraps the model server’s OpenAI-compatible /v1/audio/speech endpoint with AsyncOpenAI. There is no Auth class — the gateway has no auth layer, so create_tts passes a literal api_key="not-needed" purely because the OpenAI SDK rejects an empty string. resolve_base_url() reads MODEL_SERVER_URL and raises RuntimeError if it’s unset.

The catalog models one speaking style axis independent of voice (TTS_STYLES, 14 options — default news, plus AIR style news, TV style news, educational lecture, and the happy/sad/anger/fear/surprise/disgust emotion tags) alongside the usual voice implies language rule. TTS_SPEAKERS maps each of 23 Orpheus vendor language codes to a fixed roster of speaker names (Hindi alone has two, Kavya and Amit; the newest addition, bhb/Bhili, has seven), and picking a speaker fixes the language on the wire — the same shape as the Orpheus TTS documented for Bhashini, since it’s the same underlying roster. This is the voices-v2.json roster; see TTS models → orpheus for the older voices.json this catalog no longer targets.

Indic Nemotron (STT)

local/indic_nemotron/ hand-rolls a websockets client (stt.py) against the gateway’s raw /v1/asr/ws, since there’s no OpenAI-shaped STT endpoint to wrap. resolve_ws_url() reads MODEL_SERVER_WS_URL, same required-env-var pattern. resolve_wire_language(model, canonical) translates VoicEra’s canonical language id to Nemotron’s wire code per model, covering 27 languages including three (bho, hne, bgc) that only exist on this vendor’s roster.

What makes local different from cloud/adapter

Registration still goes through the ordinary @register_stt/@register_tts decorators, but both service.py modules make one extra call first:

from ...availability import register_local
register_local("indic_orpheus", GATEWAY_MODEL_ID)  # GATEWAY_MODEL_ID = "orpheus"

register_local(provider_id, gateway_model_id) (apps/providers/availability.py) changes what authenticated means for that provider id in GET /configuration/*:

• Cloud / adapter: authenticated = this organisation has a stored ProviderAuth entry for the provider.
• Local: authenticated = the model server’s GET {MODEL_SERVER_URL}/models currently lists gateway_model_id in its data[].id array (10-second in-process cache; a network failure or unset MODEL_SERVER_URL reads as “not deployed,” not an error).

gateway_model_id is the model server’s own slot id (its models.yaml/folder name under model-server/<kind>/), a separate namespace from the apps/providers provider id — indic_orpheus → gateway id "orpheus", indic_nemotron → gateway id "indic-nemotron".

No local LLM provider exists. model-server/llm/qwen3.5-4b/ is status: ready in the model server’s catalog, but nothing under apps/providers/local/ wires it up, so no agent can select it yet.

Related

• Provider registry — the design, and why it is self-describing
• Provider credentials (ProviderAuth) — where the secrets live
• Adding an AI provider
• Agent configuration