Is Your Speech Recognition Still Stuck on "Word by Word"? This Open Source Project Delivers a 1-Hour Meeting Transcript in 20 Seconds

You finish a 2-hour stand-up meeting, drop the recording into Whisper, and wait 9 minutes for the transcript. 9 minutes — enough time to brew a coffee and scroll through 3 tweets.

This isn't a network issue; it's an architecture issue. Whisper uses autoregressive decoding, spitting out words one by one, like reading a script aloud. Meanwhile, Alibaba Tongyi Lab's FunASR uses a non-autoregressive architecture that outputs the entire text at once, achieving 170x real-time speed on GPU and 17x real-time on CPU.

In plain English: What Whisper does on GPU, FunASR can match on CPU.

A Single Chart Redefines "Good Enough"

Data source: FunASR official benchmark, 184 Chinese long audio files (192 minutes total), tested on a single NVIDIA H100. SenseVoice's Chinese CER is only one-third of Whisper's, and its speed is 12 times faster.

What Can It Actually Do?

More than just speech-to-text. SenseVoice produces four outputs in a single forward pass:

1. Automatic Speech Recognition (ASR) — 50+ languages, auto language detection
2. Language Identification (LID) — Tells you if it's Chinese, English, Cantonese, etc.
3. Speech Emotion Recognition (SER) — 7 emotion labels: happy, sad, angry, neutral, etc.
4. Acoustic Event Detection (AED) — Applause, laughter, background music, coughs, sneezes

Run it with one line of code:

from funasr import AutoModel
from funasr.utils.postprocess_utils import rich_transcription_postprocess

model = AutoModel(model="iic/SenseVoiceSmall",
                  vad_model="fsmn-vad",
                  spk_model="cam++")

result = model.generate(input="meeting.wav")

Output includes speaker tags, timestamps, and punctuation:

[00:00.4 → 00:03.8] Speaker 0: Let's discuss the Q3 plan.
[00:04.2 → 00:07.1] Speaker 1: Sure, I have three points to make.

VAD segmentation, speech recognition, punctuation restoration, and speaker diarization — all in one model, no need to attach pyannote like Whisper.

Want to deploy an API? One command starts the server:

funasr-server --device cuda

It automatically exposes an OpenAI-compatible interface, ready to connect with Claude, Cursor, Dify, LangChain, etc.

How to Choose Among the Three Models?

Though the original article only mentions SenseVoice, FunASR is actually a toolbox with three main models, each excelling in different areas:

One-sentence selection: Use Fun-ASR-Nano if you have GPU for the best, SenseVoice if no GPU, and Paraformer for pure Chinese with hotwords.

Another notable update: FunASR recently introduced a llama.cpp-style single binary runtime — no Python environment needed. Just one executable file, similar to whisper.cpp, but with 3x lower Chinese CER. This is a killer feature for edge devices and embedded scenarios.

Reality Check: Don't Be Blinded by Speed Alone

GPU Memory May Explode

Some developers reported CUDA out of memory when processing short video audio with SenseVoice — seemingly 10-20 minute files but with maliciously crafted audio streams causing the model to attempt allocating 64.81 GB of VRAM (GPU only has 24 GB). This is especially dangerous in batch processing scenarios.

Solution: Use the ONNX quantized version (funasr-onnx). After Int8 quantization, the model runs on <4 GB VRAM. Reduce batch_size from default 10 to 2 to trade some throughput for stability.

Batch Inference Still Unstable

GitHub Issues report that the SenseVoice ONNX version returns empty results on CPU when batch_size > 1, and the second batch crashes on GPU V100. For production, use batch_size=1 — stability matters more than speed.

Emotion Recognition Is Not Universal

SenseVoice's emotion labels cover only 7 types (happy/sad/angry/neutral/fear/disgust/surprise), and cannot handle finer-grained emotions like anxiety, confusion, or anticipation. The team admits that acoustic event detection still lags behind specialized models — it's an all-rounder, not a world champion in every category.

Don't Overgeneralize the "Home Advantage" in Chinese

The 170x speed and 7.81% CER are based on Chinese benchmarks. In English or low-resource languages, Whisper still holds advantages — after all, its 57-language coverage is no joke. If your primary language is English, don't rush to switch.

Minimum Hardware Requirements

This is what most people care about: Can my current computer run it?

CPU Solution (SenseVoice / Paraformer)

4 cores, 8 GB RAM, no dedicated GPU — it runs. A 1-hour meeting takes about 3.5 minutes to transcribe, already beyond real-time.

GPU Solution (Fun-ASR-Nano flagship / high-concurrency deployment)

Minimum 4 GB VRAM (using ONNX Int8 quantization), but for production, start at 16 GB. NVIDIA T4 (16 GB, cheap second-hand) is the best value.

Edge Device Solution

SenseVoice ONNX quantized version also runs on ARM development boards like RDK X5 — friendly for IoT and embedded scenarios.

Who Should Use It?

• Meeting note takers: Record and get transcripts with speaker labels and emotions, no more guessing who said what
• AI app developers: Build your own ASR service, save cloud API costs, seamless integration via OpenAI-compatible interface
• Video creators: Use the companion FunClip tool to automatically locate spoken segments by keyword, one-click editing
• Data compliance officers: Deploy locally, data never leaves, essential for finance and healthcare
• Tech enthusiasts: An old laptop with 4 cores and 8 GB RAM is enough, zero cost to try

Final Thoughts

Whisper opened the door to open-source speech recognition, but its autoregressive architecture imposes a ceiling on speed. FunASR's non-autoregressive approach fundamentally changes the game — not by making the horse faster, but by switching to a high-speed train.

Of course, high-speed trains have their own problems: unstable batch inference, limited emotion granularity, and smaller language coverage compared to Whisper. But for Chinese scenarios, it is the best open-source deployable solution — bar none.

MIT license, free to use. Repository: github.com/modelscope/FunASR

FunASR is an open-source project by Alibaba Tongyi Lab, MIT licensed. Benchmark data in this article comes from official tests; actual speeds may vary depending on hardware and audio quality.