Skip to main content

How to run OpenThai 2.0 ASR Streaming on Windows, Mac and Linux

· 8 min read
Kobkrit Viriyayudhakorn
Kobkrit Viriyayudhakorn
CEO @ iApp Technology

OpenThai 2.0 ASR Streaming, which we released this week, writes Thai speech down while people are still talking. This guide shows how to run it on your own computer: install it, transcribe a recording, and get live captions from a microphone. Your audio never leaves your machine.

Three ways to use it​

  • No code: Chinda Bot. Chinda Bot, our free desktop app for Mac, Windows and Linux arm64, has a microphone button: press it, speak Thai, and the speech is transcribed on your computer by this model.
  • On your own machine, from your own code: the model with its small server, as this guide shows.
  • As an API: OpenThai ASR Realtime runs the same model behind a WebSocket, free during its launch, and Chinda ASR Pro v2 is our more accurate service for recordings.

What you need​

SystemLinux; Windows 10 or 11 through WSL 2; or a Mac with Apple silicon
ProcessorA CPU is enough. An NVIDIA GPU on Linux is much faster; on a Mac the model runs on the CPU
MemoryAbout 6 GB free (the server peaked at about 6 GB on our Mac)
Download2.6 GB for the model, plus the Python packages: 1.4 GB on our Mac, more on Linux, where PyTorch brings NVIDIA's GPU libraries

What we tested. Every step below was run on a MacBook Pro with M3 Max, on its CPU. Our team runs the same server on Linux with NVIDIA H100 GPUs and on Linux CPUs. We have not yet tested Windows ourselves.

1. Install​

On Ubuntu or Debian, install the tools, then uv, which installs Python and the packages:

sudo apt install git curl ffmpeg libportaudio2
curl -LsSf https://astral.sh/uv/install.sh | sh

Open a new terminal, then:

mkdir thai-asr && cd thai-asr
uv venv asr --python 3.12
source asr/bin/activate
uv pip install torch==2.13.0 --index-url https://download.pytorch.org/whl/cu130
uv pip install Cython packaging
uv pip install --extra-index-url https://download.pytorch.org/whl/cu130 --index-strategy unsafe-best-match \
  "nemo-toolkit[asr] @ git+https://github.com/NVIDIA-NeMo/Speech.git@cf724ac337d1ebc7d0dda1e23fb80916f52927a5"
uv pip install fastapi uvicorn python-multipart websockets huggingface_hub sounddevice

These are the packages our team uses. They work with or without an NVIDIA GPU.

The environment keeps these packages apart from the rest of your computer. In every new terminal, go to the thai-asr folder and run source asr/bin/activate again.

2. Download the model​

The same command on every system. It fetches the model file and its server, 2.6 GB:

hf download iapp/OpenThai2.0-ASR-Streaming --include "serve/*" --include "*_raw.nemo" --local-dir OpenThai2.0-ASR-Streaming

3. Start the server​

cd OpenThai2.0-ASR-Streaming/serve
CHINDA_DEVICE=cuda CHINDA_PRELOAD=1 python -m uvicorn app:app --port 18081

Without an NVIDIA GPU, leave out CHINDA_DEVICE=cuda.

The first start takes a minute or two. The server is ready when it prints Application startup complete. Leave it running and open a second terminal for the next steps.

4. Transcribe a recording​

In the second terminal, in the thai-asr folder with the environment activated:

curl -F file=@meeting.wav -F mode=standard http://127.0.0.1:18081/v1/transcribe

The answer is JSON; the text is in display_text, with a space between sentences.

  • Long recordings are cut where nobody speaks for 0.7 seconds, and each piece is read on its own; segments lists them. Keep it that way: read as one long stream, the model drops some of the words that follow a quiet pause.
  • mode: standard for read speech, news, lectures and dictation; conversation for casual talk and video clips; meeting adds speaker labels, from a second model that downloads on first use.
  • Other fields: lang=th-mix writes English words in Latin letters, and latency_ms is 80, 320, 560 (the default) or 1120.
  • Formats: WAV and FLAC are read directly; MP3 and other formats need ffmpeg, installed in step 1.

On our M3 Max's CPU, a 7.6-second clip took 2.2 seconds and a 41-second recording 5.9 seconds, about 3 to 7 times faster than real time. On one NVIDIA H100, files go about 257 times faster than real time.

5. Live captions from your microphone​

Save this as live_mic.py in the thai-asr folder:

live_mic.py
"""Live Thai captions from your microphone with OpenThai 2.0 ASR Streaming.

Start the server first, then:  python live_mic.py      (Ctrl+C to stop)
"""
import json
import queue
import sys
import threading

import sounddevice as sd
from websockets.sync.client import connect

SERVER = "ws://127.0.0.1:18081/v1/live"
chunks = queue.Queue()


def on_audio(data, frames, time, status):
    chunks.put(bytes(data))          # 16 kHz, mono, 16-bit: what the server expects


def show(ws):
    for message in ws:
        m = json.loads(message)
        if m["type"] == "partial":
            print("\r\033[K" + m["text"], end="", flush=True)          # the sentence so far
        elif m["type"] == "final":
            print("\r\033[K" + m["display_text"], flush=True)          # one line per pause
        elif m["type"] == "done":
            return


with connect(SERVER) as ws:
    ws.send(json.dumps({"mode": "standard", "lang": "th-TH", "latency_ms": 560}))
    ws.recv()
    reader = threading.Thread(target=show, args=(ws,), daemon=True)
    reader.start()
    with sd.RawInputStream(samplerate=16000, channels=1, dtype="int16", blocksize=1600, callback=on_audio):
        print("Listening. Press Ctrl+C to stop.", file=sys.stderr)
        try:
            while True:
                try:
                    ws.send(chunks.get(timeout=0.5))   # a short wait, so Ctrl+C works on every system
                except queue.Empty:
                    pass
        except KeyboardInterrupt:
            pass
    ws.send("end")
    reader.join(timeout=30)

Run it while the server is running, and speak:

python live_mic.py

The sentence so far appears as you speak, and each pause finishes it on a line of its own. Press Ctrl+C to stop.

  • Mac: the first time, allow Terminal to use the microphone (System Settings, Privacy & Security, Microphone).
  • Windows: run this script in Windows itself, not in Ubuntu, because Windows has the microphone. Install Python for Windows from python.org, then in Command Prompt run py -m pip install sounddevice websockets and py live_mic.py. The server in Ubuntu is reachable at the same address.

On our M3 Max's CPU, live captions keep up comfortably at 560 and 320 ms. At 80 ms the Mac works at about the speed of speech, with no margin for continuous talk, so keep 80 ms for a GPU. Change latency_ms in the script to try them: lower answers sooner, higher is more accurate.

Good to know​

  • The server has no password. Keep it on your own computer, as here, or behind your own gateway.
  • The output has no spaces between Thai words and almost no punctuation. Numbers usually come out as digits, and hesitation sounds such as เอ่อ are usually left out.
  • It is a Central Thai model. It is much less accurate on the Isan dialect and with microphones far from the speakers, and it has not been measured on telephone audio.
  • Names and rare words are spelled from how they sound; this release has no custom vocabulary.

The model card has every option, the benchmark and the full list of limits. For a CPU without Python, the ONNX version runs with ONNX Runtime alone.