• Vietnamese voice
  • Vietnamese infrastructure
  • Data stays in Vietnam

SotaSpeech: Vietnamese, heard the way a Vietnamese speaker hears it

SotaSpeech is fine-tuned for Vietnamese, targeting better recognition of pronunciation nuances and Northern – Central – Southern regional accents. Bringing speech recognition, speaker separation and time alignment together, SotaSpeech powers NoteX, RoomX and Saydi, with the service running on infrastructure in Vietnam for stronger data control and security.

Try it onlineSee the benchmark

Features

Transcription and diarization in one pass

Speaker labels, segment timestamps and punctuation come out of the same decode. Nothing is stitched together afterwards, so a transcript does not lose the thread when the conversation changes hands.

128k-token context, hours of audio

Long files are split, transcribed and re-joined automatically, and the window is wide enough to keep the surrounding conversation in view — one continuous transcript, not a pile of chunks.

Tuned for Vietnamese, 50+ languages supported

Fine-tuned on real-world Vietnamese audio, with 50+ languages supported. Hotwords, OpenAI-compatible endpoints and an export to SRT / VTT are all part of the service.

Benchmark

Results on three real-world cases (clean / meeting / dialect), run and scored by the project's own evaluation team — lower is better.

  • SotaSpeech8.37
  • Amazon Transcribe10.44
  • Gipformer 1.514.00
  • VietASR / Zipformer17.64
  • PhoWhisper21.20
  • ChunkFormer RNN-T Large Vie22.40
  • Qwen3-ASR 1.7b23.99
  • Parakeet-ctc-0.6b-Vietnamese24.97
  • BuzzASR Vietnamese27.99
  • Qwen3-ASR 0.6b30.15
  • Whisper Large v333.40
  • Cohere Transcribe33.98
  • Whisper Large Turbo41.33
  • Voxtral Small 24B 2507 STT77.60
Lower is better. Bars are normalised to the best score on the selected metric.

Results table

#ModelCleanNWERMeetingNWERDialectNWER
1SotaSpeech4.468.377.52
2Amazon Transcribe5.2010.448.35
3Gipformer 1.58.5014.009.52
4Qwen3-ASR 1.7b15.0423.9910.33
5PhoWhisper12.1621.2013.37
6Parakeet-ctc-0.6b-Vietnamese13.9624.9716.48
7ChunkFormer RNN-T Large Vie12.1722.4014.51
8Qwen3-ASR 0.6b18.7130.1513.71
9Whisper Large v322.8533.4016.58
10BuzzASR Vietnamese19.9327.9923.53
11Whisper Large Turbo31.4141.3324.35
12Cohere Transcribe29.0633.9827.65
13Voxtral Small 24B 2507 STT21.1877.6078.96
14VietASR / Zipformer10.4117.6422.06

Scope of use

The model jointly performs transcription, speaker attribution and timestamp prediction, producing time-aligned segments with absolute timestamps on the clip timeline. It is built for meeting minutes, call analytics and long-form media processing.

  • Not for covert recording or monitoring, and not as the sole basis for a legal, medical or HR decision without a human check. Timestamps are per segment, not per word.

Architecture

SotaSpeech ASR architectureThe serving path runs across the top: ffmpeg decode, voice activity detection, a request gate and the sgl-omni server, which takes the whole clip. Then the model runs in two rows. A 24-layer Whisper-Medium audio encoder emits one hidden state per frame; a VQAdaptor merges four frames in time and projects them into the language model hidden size. masked_scatter writes those frames into the text embeddings at the audio placeholder token, and a Qwen3-0.6B decoder carrying the Vietnamese LoRA emits one stream of timestamps, speaker tags and words. The service then parses that stream, collapses repeats and segments it.SotaSpeech ASRVietnamese ASR · timestamped · speaker-awareuploadffmpeg decode16 kHz · mono · f32VADFireRedVAD | RMSRequest gate4 concurrentsgl-omniT=0 · 8192 tokwhole clip · no chunkingWhisper–Medium audio encoder80 mel bins · 24 layers · d_model 102416 heads · FFN 4096 · log-mel in · GELU24× Self-Attention Layer30 s window · 1500 positions · 50 frames/sencoder hidden states (B,T,1024)…VQAdaptor4× Time Merge(B,T,1024) reshape to (B,T/4,4096)ProjectionLinear 4096→1024 · SiLU · Linear 1024→1024LayerNorm · ε 1e−650 → 12.5 tokens/smaps encoder dim to LM hidden 1024masked_scatterinto text embeddingsaudio placeholder token 151671audio frames written at the placeholderQwen3–0.6B decoder28 layers · d 1024 · GQA 16/8head_dim 128 · SiLU · 131072-token contextDecoder Self-AttentionRoPE θ 1e6 · RMSNormVietnamese LoRArank 256 · checkpoint 108ktied embeddings · lm_head to 151936 vocabjointly: words + speaker + timeone autoregressive token stream[0.11][S01]Goodmorning![1.03][1.11][S02]Morning,guys![1.34]Parse[t] ([Sxx])? text [t]Collapse repeatsSegment[ ]timestampspeakerwordsspeaker tag: base only · VI adapter emits noneno CAPU · model emits cased text

Served base model: a Whisper encoder with a Qwen3 decoder and a long-context design. The Vietnamese adapter and surrounding service are this project’s own.

Demo

Speaker-coloured transcript under each clip — click a line to seek. Every recording is published by its owner under the licence credited on the card.

Hậu trường phim — nhiều người nóiTiếng Việt · Hậu trường
subtitles appear here during playback
Transcript51 segments

Diễn viên và đoàn phim nói chuyện tự nhiên, luân phiên nhanh, có tiếng ồn hiện trường — ca khó nhất cho phân tách người nói.

Source: Galaxy Studio · CC BY 4.0 · transcript: SotaSpeech V2.5

Phỏng vấn diễn viên — 2–3 ngườiTiếng Việt · Phỏng vấn
subtitles appear here during playback
Transcript23 segments

Phỏng vấn ngắn, thu gọn gàng, ít người nói — dễ đọc kết quả diarization hơn bản hậu trường.

Source: Galaxy Studio · CC BY 3.0 · transcript: SotaSpeech V2.5

Bản tin dịch vụ công — phóng viên + người dânTiếng Việt · Bản tin
subtitles appear here during playback
Transcript28 segments

Phóng sự truyền hình về thủ tục hành chính: lời dẫn xen phỏng vấn người dân và cán bộ. Nhiều từ vựng nghiệp vụ, sát với transcription thực tế.

Source: Đài PTTH Thanh Hoá · CC BY 3.0 · transcript: SotaSpeech V2.5

Phóng sự hành chính — thu hiện trườngTiếng Việt · Bản tin
subtitles appear here during playback
Transcript35 segments

Giọng miền Trung, thu tại quầy giao dịch nên có tiếng nền — kiểm tra độ bền với nhiễu.

Source: Đài PTTH Thanh Hoá · CC BY 3.0 · transcript: SotaSpeech V2.5

Thuyết minh khoa học — 30 giâyTiếng Việt · Khoa học
subtitles appear here during playback
Transcript7 segments

Thuyết minh một giọng, đọc rõ, chỉ 30 giây — mẫu chạy nhanh khi cần thử lại đường ống.

Source: ZDF/Terra X (R. Marel, I. Gesang, SpiegelTV, Jochen Schmidt) · CC BY 4.0 · transcript: SotaSpeech V2.5

Hướng dẫn y tế — giọng đọc studioTiếng Việt · Studio
subtitles appear here during playback
Transcript30 segments

Thu trong studio, một người nói rõ ràng — mốc so sánh sạch với các phóng sự thu ngoài hiện trường.

Source: San Francisco Government Television (SFGovTV) · CC BY 3.0 · transcript: SotaSpeech V2.5

Interview — 2 speakersEnglish · Interview
subtitles appear here during playback
Transcript17 segments

Question-and-answer exchange between an interviewer and a guest.

Source: Kara Fern (ZimtHaus) · CC BY 3.0 · transcript: SotaSpeech V2.5

Media day Q&A — multi-speakerEnglish · Press
subtitles appear here during playback
Transcript62 segments

Press scrum: several reporters questioning one subject, with crosstalk.

Source: MMA Fight - Official · CC BY 3.0 · transcript: SotaSpeech V2.5