On-device speech-to-text benchmark: Parakeet vs Whisper vs Gemma, tested on a laptop

We ran every speech model Walkie ships — plus Gemma 4's audio mode — on one ordinary laptop, then checked the results against the best-known public benchmark of real human speech. Here is what we found, what we run, and why.

What Walkie runs

Walkie has two modes. On-Device Mode runs a speech model on your computer and makes no network request. Cloud Mode sends the audio to a transcription provider and adds an AI cleanup step for punctuation, filler words and formatting. You switch between them whenever you like.

We retired five models this year. SenseVoice, Breeze ASR and Moonshine Base were almost never chosen; Parakeet V2 and Whisper Large are covered by V3 and Turbo, which the tests below explain. Anyone who already has one keeps it. A shorter list of models that each earn their place beats a long one you have to research.

How to read the numbers

Word error rate is the standard accuracy score: out of every 100 words, how many came out wrong — swapped, dropped or added. 5% means one mistake in twenty words. Lower is better. Both our test and the leaderboard strip punctuation and capitals before scoring, so a model is not penalised for writing “4:30” where another writes “four thirty”.

Speed is shown two ways. Our test reports the real time a laptop took per clip, which is what you feel. The leaderboard reports how many seconds of audio a model gets through per second on a data-centre GPU — useful for comparing models with each other, not for predicting your laptop.

Which on-device model is fastest? Every model, one laptop

We read 20 realistic dictation phrases in 4 English accents — 80 clips, 7 minutes of audio — through each model on a 14-inch MacBook Pro, Apple M4 (10-core CPU, 10-core GPU), 24 GB. Every model ran on the same engine builds the Walkie app uses, loaded once and warmed up once, the way the app keeps a model ready between dictations.

What stood out

• Parakeet is in a speed class of its own. Both versions took about 0.16 s per clip — around 33 times faster than real time. Whisper Large V3 Turbo took 2.9 s and Whisper Large V3 took 4.0 s. Large was slower than Turbo without being more accurate here, which is why Walkie offers Turbo instead. When you dictate dozens of times a day, that gap is the difference between text that appears and text you wait for.
• Whisper is the most accurate on clean audio, at 2.2% for Turbo. Parakeet V2 made 3.1% errors, tying Whisper Medium while running 13 times faster. V2 is English-only, so Walkie ships V3: the same speed, 24 more languages, and about a point more error in English.
• Parakeet's misses were mostly jargon. “Kubernetes” came out as “Cuban Eats” and “amoxicillin” was misspelled; Whisper got both. This is what Walkie's custom dictionary is for: add the word once and it comes out right in either mode.
• Whisper Small is the middle ground — 3.5% errors in 0.75 s, in 99 languages.
• Moonshine is built for US English. Moonshine V2 Medium scored 1.7% on the US voice and 16.1% on the British one; Tiny reached 42.7% on the Australian voice.

A real voice through a laptop microphone

Computer voices are a fair way to compare speed and an unfair way to judge accuracy. So we ran the same 20 phrases again, read aloud by one person into a MacBook Pro built-in microphone, trimmed the way Walkie trims a dictation.

• Every model got three to six times worse. Whisper Large V3 Turbo went from 2.2% to 12.9%; Parakeet V3 from 4.4% to 13.6%. A benchmark on clean audio tells you about the benchmark.
• The gap between Whisper and Parakeet closed. On a real voice they are within a point of each other — 12.9% against 13.6% — while Parakeet answers in 0.13 s and Whisper in 2.9 s.
• Gemma's clean-audio lead did not survive. Gemma 4 E4B scored 2.4% on computer voices and 16.1% on a real one, behind both Whisper and Parakeet.
• Apple's built-in recognizer is the fastest thing we measured at 0.10 s per clip, and lands between Parakeet and Gemma on accuracy.

Does AI cleanup help, and what does it cost?

A transcript is not yet writing. People say “send it to John, no wait, Sarah” and mean “send it to Sarah”; they say “four thirty PM” and mean “4:30pm”. So we scored the same 20 real-voice clips a second way: against the text the speaker meant, plus 12 pass-or-fail checks of exactly those things. The cleanup step runs on the computer too — two sizes of Google's Gemma 4, and Apple's built-in model for comparison.

• Cleanup is worth about four errors in every hundred words, for roughly half a second: Parakeet alone scored 16.1%, and 12.1% with Gemma 4 E2B behind it.
• The most accurate setup was also the slowest. Whisper Large V3 Turbo with Gemma 4 E4B passed 10 of 12 checks but took 3.8 s — almost all of it Whisper. Swapping in Parakeet gives up a point or two and answers in under a second.
• Not every on-device model can do this job. Apple's built-in model made the text worse than no cleanup at all (24.9%): it dropped words the speaker said, turned a spoken list into numbered points, and refused one ordinary sentence as unsafe.

Is cloud dictation faster than on-device?

We assumed it would be. It is not. Walkie logs where the time goes on every Cloud Mode dictation, and across 138 real ones the median was 0.81 s from releasing the key to having the text: 0.24 s transcribing, 0.26 s on AI cleanup, and 0.24 s simply travelling there and back.

On the same laptop, on-device transcription with Parakeet took 0.13 s–0.35 s. Transcribing takes about as long in either place; the cloud then adds the trip there and back. For anything as short as a dictation, a fast local model is hard to beat.

How accurate are on-device models on real speech?

The Hugging Face Open ASR Leaderboard scores 66 speech models on the same real recordings — meetings, earnings calls, audiobooks, podcasts. It is the closest thing the field has to a neutral referee, and it includes the paid cloud engines. The table below is a selection; the full board is linked in the sources.

What the leaderboard says

• The cloud engines lead. Zoom, Microsoft and ElevenLabs score between 3.59% and 3.97% — better than anything you can run yourself. The cost is that your audio lives, however briefly, on their servers.
• Parakeet is the most accurate model in its speed class. Parakeet V2 and V3 score 4.7% and 4.86% while processing audio about 6,025 times faster than real time on the benchmark's GPU. Every commercially usable model that is more accurate than Parakeet V2 is at least 2.9 times slower; the most accurate open model, Alibaba's Qwen3-ASR 1.7B (4.31%), is about 7 times slower and three times larger. Parakeet V3 ranks 9 of the 17 shown here, ahead of every Whisper model.
• Whisper Large V3 wins on European languages. It averages 3.41% across six languages — the best of any open model — against 3.89% for Turbo and 4.23% for Parakeet V3. Walkie offers Turbo rather than Large: on our laptop it was faster and at least as accurate, and on this set it trails Large by about half a point.
• Clean-audio results do not transfer. Gemma 4 E4B, which scored 2.4% on our computer voices, scores 9.51% on real speech — about twice Parakeet's error rate.

Why not use a chatbot model like Gemma or Voxtral?

Models such as Google's Gemma 4 and Mistral's Voxtral can listen as well as read, which makes it tempting to use one model for everything. For dictation, three things get in the way:

• Accuracy on real speech. On the leaderboard, Gemma 4 E4B scores 9.51% and E2B 11.64%, against 4.86% for Parakeet V3. Voxtral Mini 3B does better at 5.54%, but runs at 181× against Parakeet's 6,076×.
• A 30-second limit. Google documents a maximum of 30 seconds of audio per request for Gemma. Long dictations have to be cut up, and every cut is a place to lose a word.
• Size and speed. Gemma 4 E4B is a 5.2 GB download and took 1.3 s per clip on our laptop. Parakeet V3 is 478 MB and took 0.16 s.

Language models are excellent at a different job: understanding and rewriting text. The strongest setup pairs a dedicated speech model that listens with a language model that writes — which is exactly how Walkie's Cloud Mode works.

What about Nemotron 3.5 and Moonshine?

NVIDIA Nemotron 3.5 is a streaming model: words appear while you are still talking, in up to 40 languages. It scores 7.88% on English and 7.51% on the European set — behind Parakeet on both. Walkie shows your text when you finish speaking, and Parakeet finishes a whole clip in 0.16 s, so streaming would not make dictation feel faster.

Moonshine is tiny and quick. It stays in Walkie for US English on older computers, where a 31 MB download matters more than accents it was not trained on.

Cloud Mode: what runs, and why

Cloud Mode transcribes with Whisper Large V3 Turbo on Groq's hardware and switches to full Whisper Large V3 when it suspects Turbo has translated instead of transcribed. An AI cleanup step then adds punctuation, removes filler words and formats the text for the app you are typing into. Cloud requests run under zero data retention, and nothing is used to train AI models.

Whisper Turbo is not the most accurate engine on the leaderboard — it scores 6.36% to Azure's 3.81%. We use it because it is fast, covers 99 languages, and the cleanup step does what no raw transcript does: turns speech into writing. If raw English accuracy is what you are after, Parakeet in On-Device Mode scores better than Whisper Turbo on the same benchmark — and never leaves your computer.

How Walkie compares to Wispr Flow and others

The honest version: a cloud-only tool like Wispr Flow may well run an engine as accurate as the leaders above — it just does not say, and every word you dictate goes to its servers to find out. Apple Dictation is free and already installed. What Walkie adds is the choice: the fastest accurate local model by default, Whisper when you need 99 languages or rare words, the cloud when you want polish — and the numbers to decide with.

For a feature-by-feature look at the apps that run these models on your computer — Handy, VoiceInk, MacWhisper, Superwhisper and more — see the best local dictation apps, compared.

Which model should you pick?

• You dictate in English or another of 25 European languages and want it instant: Parakeet V3, Walkie's default.
• Other languages, or lots of names and technical terms: Whisper Small if you want speed, Whisper Large V3 Turbo if you want accuracy — and add the words you use to your custom dictionary.
• An older or low-memory computer: Whisper Small, or Moonshine for US English.
• The best-written result, and the cloud is fine: Cloud Mode.

Method and sources

Laptop test, 2026-09-19: 80 clips (2.4–8.2 seconds each) generated with the Samantha (US), Daniel (UK), Rishi (India), Karen (Australia) voices, 16 kHz mono. Every model used the exact file Walkie downloads, including Walkie's quantized Whisper Medium and Large, which is why our Whisper numbers are Walkie's, not the full-precision models'. Each model was loaded once, warmed with one clip, then timed per clip; the table shows the median. Gemma ran through llama.cpp with Google's recommended transcription prompt. Scoring used the Whisper English normalizer — the one the leaderboard uses — and word error rate across all clips.

Leaderboard figures were retrieved 2026-09-19 from the published English results and multilingual results files. Soniox and Deepgram are not on the leaderboard, so they are not in these tables.

The 20 test phrases

1. um so I think we need to uh update the docs before Friday
2. So basically the plan is to ship the new version on Friday.
3. Send it to John, no wait, send it to Sarah.
4. It costs fifty, no, sixty dollars.
5. Let's meet Monday, actually Tuesday, at 4:30 PM.
6. I like the blue one better than the green one.
7. Turn right at the light, then go two blocks.
8. Summarize this for me and tell me what the weather is.
9. Send it to example.com, and the budget is $25,000.
10. Hey John, just wanted to say thanks for the call today. Talk soon, Adam.
11. Hey Sarah, can you move our Thursday sync to 3 PM and send the updated deck to the whole team before then?
12. Quarterly revenue grew 18 percent to 2.4 million dollars, mostly from the enterprise tier.
13. Please add Kubernetes, PostgreSQL, and the Redis cluster to the migration checklist for next sprint.
14. My flight lands at 7:45 on Friday, so I'll probably miss the first half of the offsite.
15. Remind me to call Dr. Patel about the lab results and to renew the prescription for amoxicillin.
16. I think the second option is better because it's cheaper, faster to build, and easier to maintain over time.
17. Can you summarize the three main risks in the proposal and suggest one mitigation for each?
18. Order two large pizzas, one margherita and one pepperoni, and have them delivered to 1200 Market Street.
19. We need to finish the onboarding redesign, fix the login bug on Android, and ship version 2.1 by the end of the month.
20. Thanks for the thoughtful feedback on the draft. I've incorporated most of it, and I'll send a new version tomorrow morning.

Questions