NVIDIA’s Nemotron 3.5 Transcribe Offers Open‑Weight Multilingual Speech‑to‑Text, Speaker Labels and Translation
THE SHIFT
Do you need a single tool that can turn a multilingual meeting recording into a clean transcript, label who spoke, and even translate the content on demand? Until now developers have had to stitch together separate speech‑to‑text, diarization and translation services, each with its own API, latency and cost structure, making integration time‑consuming and budgets hard to predict.
NVIDIA’s Nemotron 3.5 Transcribe collapses that stack into one checkpoint you can steer with natural‑language prompts for raw, cleaned or translated output. The model is open‑weight and hosted on Hugging Face and ModelScope, so anyone can download the weights, run it locally or launch a cloud endpoint without a proprietary licence. Granary v2 supplies a curated 463 k‑hour multilingual corpus that developers can fine‑tune for niche vocabularies such as medical terminology.
THE PROOF
NVIDIA reports that Nemotron 3.5 Transcribe can transcribe speech in 40 languages, translate 31 of them to or from English, and attribute speakers in 13 languages, all from a single checkpoint. On the Hugging Face Open ASR Leaderboard it leads all open‑weight models with an average English word‑error rate of 5.33 %, and a mean 4.62 % WER across seven multilingual test sets.
The model also posts the lowest diarization error rate on David AI’s 21‑language leaderboard and ranks first in VoiceArena’s 14‑system comparison, confirming its speaker‑attribution strength. In a medical fine‑tuning case study, adapting the model with domain data cut word‑error rate on held‑out speakers by 45.8 % relative to the best prompt, demonstrating that prompt‑level control can be sharpened with modest data.
THE CATCH
Nemotron 3.5 Transcribe is designed for recorded audio, not live streaming, and each inference is limited to recordings up to one hour with a maximum of eight speakers. While the model can run on a single GPU for modest workloads, the high‑throughput quantized version that enables processing 100 hours in under three minutes is still pending release.
Running the full model at peak speed requires a recent NVIDIA GPU such as RTX Spark, DGX Spark or Jetson AGX Thor, or a Blackwell‑class data‑center GPU for large‑scale serving. Although the model is open‑weight and free of licensing fees, the compute cost for fine‑tuning or large‑batch inference can be significant for teams without access to NVIDIA’s accelerated libraries.
YOUR MOVE
Begin by testing the live demo on Hugging Face or ModelScope to see how a single natural‑language prompt can produce a transcript, speaker labels or a translation. If the baseline meets your needs, you can self‑host the checkpoint with vLLM or SGLang‑Omni, or use a hosted endpoint from providers such as Baseten, DeepInfra or Simplismart for quick integration.
For applications that require higher accuracy on specialized terminology, download the Granary v2 manifests and follow NVIDIA’s NeMo Speech fine‑tuning guide. Starting with about 100 labeled clips, you can train a LoRA adapter to adapt the model to your domain, then deploy the adapted checkpoint through the same serving stack, achieving better word‑error rates without rebuilding the entire pipeline.
Nemotron 3.5 Transcribe opens powerful multilingual speech processing to anyone, but hardware and fine‑tuning remain required for domain‑specific precision.
Sources & further reading
Published by The Breakdown. How we report · Our source desk.
