Streaming synthesis

What you can build

Streaming synthesis separates how much text you have from when audio starts. You can begin playing the first sentence while the rest is still being written.

• Speak an LLM’s output as it generates. Buffer tokens into sentences, push each one, and play audio continuously. Time to first sound is one sentence rather than one full answer.
• Long-form narration without a long wait for the first byte.
• Live voice agents where a caller hears a response start immediately.
• Multi-turn sessions. One connection carries many text frames, so a whole conversation’s audio flows over a single socket.

How it works

One WebSocket to wss://tts.navana.ai/v1. You open with a hello frame that locks in the session’s language, voice, and format. Then you send text frames as text becomes available, and finish with end.

Send the key as an X-API-Key header on the upgrade, or as an auth_token field in the hello frame, as the examples below do.

sequenceDiagram
participant C as Your backend
participant B as Bodhi
C->>B: WebSocket upgrade
C->>B: {"type":"hello", ..., "auth_token":"..."}
B-->>C: {"type":"ready","sample_rate":24000,...}
loop per text frame
  C->>B: {"type":"text","seq":n,"target_text":"..."}
  B-->>C: binary audio chunks
end
C->>B: {"type":"end"}
B-->>C: {"type":"done"}
B->>C: close

The protocol

1. Connect and send hello

The first frame authenticates the connection and fixes its settings. None of them can change afterwards.

import asyncio, json, os
from websockets.asyncio.client import connect

async def speak():
    async with connect("wss://tts.navana.ai/v1") as socket:
        await socket.send(json.dumps({
            "type": "hello",
            "lang": "hi",
            "voice": "default_female",
            "output_format": "24000:pcm16",
            "auth_token": os.environ["BODHI_API_KEY"],
        }))
        ...

Only auth_token is genuinely required, but send the rest explicitly. The default format is float32, and a field the server does not recognise closes the connection rather than being ignored. The full field list is in the reference.

The server replies once it is ready:

{
  "type": "ready",
  "session_id": "…",
  "protocol_version": 2,
  "sample_rate": 24000,
  "encoding": "pcm16",
  "bytes_per_sample": 2,
  "max_text_len": 10000,
  "idle_timeout_s": 180
}

2. Send text frames

One per unit of text. You own seq, so increment it yourself. The server splits your text into sentence chunks internally and streams back one audio message per chunk, and seq tags which text frame the replies belong to.

{ "type": "text", "seq": 0, "target_text": "आपके खाते में पाँच हज़ार रुपये हैं।" }

Audio comes back as binary WebSocket frames, raw samples in the negotiated format. Chunks concatenate with no seams, so append or play them directly.

3. Send end

{ "type": "end" }

The server finishes the outstanding audio and sends {"type": "done"}, then closes.

Frame reference

You send:

You receive:

Each binary chunk is preceded by an audio frame describing it. If you only want the bytes you can ignore that metadata, but do expect text frames interleaved with the binary ones rather than treating every text frame as an error.

A complete client

import asyncio, json, os
from websockets.asyncio.client import connect

URL = "wss://tts.navana.ai/v1"

async def speak(sentences, lang="hi", voice="default_female", rate=24000):
    async with connect(URL) as socket:
        await socket.send(json.dumps({
            "type": "hello",
            "lang": lang,
            "voice": voice,
            "output_format": f"{rate}:pcm16",
            "auth_token": os.environ["BODHI_API_KEY"],
        }))

        ready = json.loads(await socket.recv())
        sample_rate = ready["sample_rate"]

        for seq, sentence in enumerate(sentences):
            await socket.send(json.dumps({
                "type": "text", "seq": seq, "target_text": sentence,
            }))
        await socket.send(json.dumps({"type": "end"}))

        audio = bytearray()
        async for message in socket:
            if isinstance(message, (bytes, bytearray)):
                audio += message
                continue
            frame = json.loads(message)
            if frame["type"] == "error":
                raise RuntimeError(frame)
            if frame["type"] == "done":
                break

        return bytes(audio), sample_rate

import WebSocket from 'ws'

export function speak(
  sentences: string[],
  { lang = 'hi', voice = 'default_female', rate = 24000 } = {},
): Promise<{ audio: Buffer; sampleRate: number }> {
  const socket = new WebSocket('wss://tts.navana.ai/v1')
  const chunks: Buffer[] = []
  let sampleRate = rate

  return new Promise((resolve, reject) => {
    socket.on('error', reject)

    socket.on('open', () => {
      socket.send(JSON.stringify({
        type: 'hello',
        lang,
        voice,
        output_format: `${rate}:pcm16`,
        auth_token: process.env.BODHI_API_KEY,
      }))
    })

    socket.on('message', (data, isBinary) => {
      if (isBinary) {
        chunks.push(data as Buffer)
        return
      }
      const frame = JSON.parse(String(data))
      switch (frame.type) {
        case 'ready':
          sampleRate = frame.sample_rate
          sentences.forEach((target_text, seq) =>
            socket.send(JSON.stringify({ type: 'text', seq, target_text })))
          socket.send(JSON.stringify({ type: 'end' }))
          break
        case 'error':
          reject(new Error(JSON.stringify(frame)))
          break
        case 'done':
          socket.close()
          resolve({ audio: Buffer.concat(chunks), sampleRate })
          break
      }
    })
  })
}

package main

// go get github.com/gorilla/websocket
import (
	"encoding/json"
	"fmt"
	"net/http"
	"os"
	"strconv"

	"github.com/gorilla/websocket"
)

func speak(sentences []string, lang, voice string, rate int) ([]byte, int, error) {
	header := http.Header{} // no auth header: the key goes in the hello frame
	conn, _, err := websocket.DefaultDialer.Dial("wss://tts.navana.ai/v1", header)
	if err != nil {
		return nil, 0, err
	}
	defer conn.Close()

	if err := conn.WriteJSON(map[string]any{
		"type":          "hello",
		"lang":          lang,
		"voice":         voice,
		"output_format": strconv.Itoa(rate) + ":pcm16",
		"auth_token":    os.Getenv("BODHI_API_KEY"),
	}); err != nil {
		return nil, 0, err
	}

	var ready struct {
		Type       string `json:"type"`
		SampleRate int    `json:"sample_rate"`
	}
	if err := conn.ReadJSON(&ready); err != nil {
		return nil, 0, err
	}

	for seq, text := range sentences {
		conn.WriteJSON(map[string]any{"type": "text", "seq": seq, "target_text": text})
	}
	conn.WriteJSON(map[string]string{"type": "end"})

	var audio []byte
	for {
		msgType, data, err := conn.ReadMessage()
		if err != nil {
			return nil, 0, err
		}
		if msgType == websocket.BinaryMessage {
			audio = append(audio, data...)
			continue
		}
		var frame struct {
			Type string `json:"type"`
		}
		json.Unmarshal(data, &frame)
		if frame.Type == "error" {
			return nil, 0, fmt.Errorf("server error: %s", data)
		}
		if frame.Type == "done" {
			return audio, ready.SampleRate, nil
		}
	}
}

func main() {
	audio, rate, err := speak(
		[]string{"आपके खाते में पाँच हज़ार रुपये हैं।"}, "hi", "default_female", 24000)
	if err != nil {
		panic(err)
	}
	fmt.Println(len(audio), "bytes at", rate, "Hz")
}

Important considerations

Billing is by characters sent, not audio produced. Closing the socket early does not refund text you already sent.

Buffer to sentence boundaries. Sending a text frame per token produces choppy prosody, since the engine synthesizes what you give it. Accumulate until a sentence terminator, then send.

One voice and language per connection. hello fixes both. To change either, open a new connection.

Keep credit headroom. Usage is reported while the session runs, so a balance that empties mid-call ends the session with an insufficient_funds frame at the next frame boundary. Audio already in flight is delivered, but the call is over. Budget headroom rather than running to the floor.

Using the Python SDK

If you are writing Python, the SDK speaks this protocol for you — the hello handshake, the sequence numbers, and the end frame:

pip install bodhi-api-sdk

import asyncio, os
from bodhi import BodhiTTSClient

async def main():
    client = BodhiTTSClient(api_key=os.environ["BODHI_API_KEY"])

    async for chunk in client.stream(
        "आपके खाते में पाँच हज़ार रुपये हैं।",
        lang="hi",
        voice="default_female",
        sample_rate=8000,
        encoding="pcm16",
    ):
        print(len(chunk))      # feed this to your audio device

asyncio.run(main())

Chunks arrive in order, so the first plays while the rest is still being synthesized. Pass a list of strings instead of one to send several utterances down the same connection — that is how you speak an LLM’s sentences as they arrive:

async for chunk in client.stream(["पहला वाक्य।", "दूसरा वाक्य।"], lang="hi"):
    play(chunk)

stream_to_file(text, "speech.wav", ...) collects the whole utterance into a playable file instead. speed is deliberately absent: the hello frame rejects it, exactly as described above.

Errors

There are no HTTP statuses here past the upgrade. Failures arrive as an error frame, and every one of them is fatal: the server closes the connection immediately after.

{ "type": "error", "code": "text_too_long", "message": "…", "fatal": true, "seq": 3 }

The ones worth handling explicitly:

The full list is in the reference.

Related