Benchmark

Last updated • 3 min read

Evoglyph sells speed and locality. This page puts numbers on both, next to the alternatives developers shortlist: Wispr Flow, Superwhisper, and Apple Dictation. Our numbers are measured; competitor numbers are cited, with retrieval dates so you can check them.

How Evoglyph compares

Metric evoglyph Wispr Flow Superwhisper Apple Dictation
p50 latency 375ms for a short dictation; 970ms for a 15- to 30-second paragraph (full pipeline, cleanup on, Apple M4 Max)1 0.83s median for a 27-second passage (release to text on screen, measured by PUSHTYPE, a competing vendor)2 1.42s median for a 27-second passage (release to text on screen, measured by PUSHTYPE, a competing vendor)2 150-400ms (engine-level, as reported by Dictato)4
WER on LibriSpeech test-clean 2.55% (our harness, 75-clip subset)5 unconfirmed 2.2% on LibriSpeech (as reported by Superwhisper for its cloud S1-Voice model; split not stated)3 unconfirmed6
Fully local? Yes7 No8 Optional (local voice model plus S1-mini cleanup in Message and Mail modes; default models are cloud-hosted)3 Configurable9

Evoglyph rows are our own measurements. Competitor rows are vendor-published or third-party figures, cited below; no two products here are measured on a common harness, and the Wispr Flow and Superwhisper latency figures come from PUSHTYPE, a competing dictation vendor, on a different Mac. A cell marked unconfirmed means the vendor publishes no value and we could not triangulate one from a credible third party.

Read the latency row with scope in mind. Apple's 150-400ms is an engine benchmark, not a full pipeline. Evoglyph's figures include an AI cleanup pass that runs on-device, on by default, and come from one Apple M4 Max.1 PUSHTYPE timed a 27-second passage on a Mac mini (M4), including paste and, for Wispr Flow and Superwhisper, the network round trip.2 The closest evoglyph bucket, a 15- to 30-second paragraph, runs 970ms p50 with cleanup on: above PUSHTYPE's Wispr Flow median and below its Superwhisper median. Wispr Flow's own stated target is text within 700ms of when you stop speaking; that is a goal, not a measurement.11 Superwhisper says most S1-Voice dictations under 30 seconds appear 0.32 seconds after you stop talking, for its cloud voice model alone.3 On accuracy, Superwhisper reports 2.2% on LibriSpeech for S1-Voice against our 2.55% on LibriSpeech test-clean, on different harnesses. We found no published WER for Apple Dictation.6

Metrics only we publish

No vendor on this page publishes these, so there is nothing to compare against. Dictation engines can emit phantom text from silence or background noise; the last two rows track that failure mode.

Metric evoglyph
p95 latency 500ms for a short dictation (full pipeline, cleanup on, Apple M4 Max)1
Empty outputs on speech 0 in 100 clips5
Hallucinated text on silence 0 in 5 clips5

Why the cloud round-trip matters

Wispr Flow transcribes in the cloud, so every dictation pays for the network: TLS handshake, audio upload, server inference, text download.8 That cost is structural; no patch removes it. PUSHTYPE, a competing vendor, measured a 0.83s median for a 27-second passage, network round trip included.2 Evoglyph runs the whole pipeline on your Mac's Apple silicon, inside your process. No round-trip. Audio never leaves your Mac.7

Local alone does not distinguish us: Superwhisper can run both its voice model and its S1-mini cleanup model on-device3 and Apple Dictation has an on-device mode.9 The difference is what we publish: full-pipeline latency from real dictations at p50 and p95, a measured 2.41% overall error rate across 95 public clips,5 and an on-device cleanup pass included in the 375ms above.1

Methodology

How we measure latency

Latency is the full pipeline, from hotkey release to text in the focused app, measured across 1,379 real dictations from 2026-09-01 to 2026-11-03, on the code that shipped as version 0.3.8: one user, one machine, an Apple M4 Max.1 The pipeline runs voice-activity trimming, Parakeet TDT speech-to-text on the Apple Neural Engine, and AI cleanup, all on-device, cleanup on by default. A short dictation of five seconds or less runs 375ms p50 and 500ms p95, and all 323 short dictations in the window finished in under one second. The transcription stage alone runs about 140ms p50 and 200ms p95. Latency scales with input: a 5- to 15-second dictation runs 530ms p50 and 811ms p95, a 15- to 30-second paragraph runs 970ms p50 and 1.4s p95, and dictations over 30 seconds run 2.1s p50 and 5.7s p95, because cleanup works through more text.1 Real-world times vary by Mac and how long you speak: the figures above are an Apple M4 Max baseline, and a slower Mac or a longer dictation runs longer.

Most dictations in this window ran with vocabulary boosting, a stage that ran alongside speech-to-text. Version 0.3.8 turned it off, and our replay test of the shipped build on 2026-11-03 showed no slowdown.1 Earlier baseline: June 2026, about 600ms p50 and 1.5s p95 for a short dictation, across about 2,500 dictations on the same Mac with the same method. This baseline replaces it.

Sub-stage times do not sum to the pipeline totals; percentiles never add. The raw ASR-engine slice runs about 120ms in production,1 and our eval harness reports about 85ms on clean fixtures.5 Neither is a number you feel, so the table leads with the full pipeline.

How we measure accuracy

WER, empty-output, and silence-hallucination numbers come from a 105-clip public evaluation set: 75 clips from LibriSpeech test-clean, 20 from the VCTK corpus, 5 silence clips, and 5 vocabulary-dense clips, run on the same M4 Max.5 Reference transcripts and model output both pass through OpenAI's EnglishTextNormalizer before WER is computed, so punctuation and contraction artifacts do not inflate the error rate. Cleanup quality comes from a separate 117-fixture editorial-cleanup eval against the production prompt and LoRA bundle that ship in the app.10

Data availability

Raw per-clip results from both evals are available on request via [email protected]. We do not update this page silently: each new baseline lands as a dated entry in our internal eval/results/ archive with a new citation here. The current latency baseline is eval/results/latency-baseline-2026-11-03.md.

Sources

  1. Evoglyph full-pipeline end-to-end latency, hotkey release to text in the app, cleanup on (375ms p50 / 500ms p95 for a short dictation of five seconds or less, n = 323; 970ms p50 / 1,391ms p95 for 15 to 30 seconds; 672ms p50 / 3,748ms p95 across all 1,379 dictations; transcription stage alone 138ms p50 / 196ms p95 short; ~120ms p50 for the ASR-engine slice), measured from the production transcriptions database across 1,379 real dictations from 2026-09-01 to 2026-11-03 on an Apple M4 Max (Parakeet TDT v2 + LFM2-2.6B 4-bit cleanup, on by default; the code that shipped as 0.3.8). Replay test of the shipped build, vocabulary boosting off: eval/results/replay-2026-11-03.md. Internal measurement; app-repo eval/results/latency-baseline-2026-11-03.md. 2026-11-03. Supersedes the June 2026 baseline (~600ms p50 / ~1.5s p95 short, about 2,500 dictations; internal-docs/latency-claims-handoff.md, 2026-06-24).
  2. Wispr Flow and Superwhisper latency, release to text on screen, including paste and the network round trip: PUSHTYPE benchmark. PUSHTYPE is a competing dictation vendor and published this on its own site, not a neutral lab. Mac mini (M4, 24 GB), macOS 27.2, one 27-second recorded passage, 25 runs per app. Wispr Flow 1.6, measured 2026-10-16: 0.83s median, range 0.73-1.00s. Superwhisper 3.0.1, measured 2026-10-18 (20 runs in Default mode, 5 in Local mode): 1.42s median, range 1.33-1.81s. Retrieved 2026-11-03.
  3. Superwhisper architecture and published figures: Introducing the S1 family (2026-08-19) and language model docs. S1-mini is a local cleanup model: it "works offline, and with a local voice model nothing you dictate leaves your machine", in Message and Mail modes only, English only. S1-Language, a cloud model, is the default. The launch post reports S1-Voice, its cloud voice model, at 2.2% WER on LibriSpeech and says most dictations under 30 seconds appear 0.32 seconds after you stop talking. Retrieved 2026-11-03.
  4. Apple Dictation / SpeechAnalyzer latency, 150-400ms (engine-level): Dictato engine benchmark (published 2026-03-08): "In testing, it ranges from 150ms to 400ms". Retrieved 2026-11-03.
  5. Evoglyph WER, empty-output, and silence-hallucination numbers, plus eval-harness engine time (~85ms): internal eval results, app-repo eval/results/cutover-readiness.md and README Evaluation section (#parakeet-vs-whisperkit-105-public-clips). Hardware: Apple M4 Max, 64 GB. Eval set: 105 public clips (75 LibriSpeech test-clean + 20 VCTK + 5 silence + 5 vocab-dense). 2026-05-11.
  6. Apple Dictation WER, unconfirmed. We previously cited ~8% to the MacRumors transcription benchmark, but the article reports speed only and gives no error rate. Retrieved 2026-11-03.
  7. Evoglyph architecture (local-only, proprietary): Privacy and local-first docs page. 2026-05-11.
  8. Wispr Flow cloud transcription: wisprflow.ai/privacy: "transcription always happens in the cloud to provide the best speed and accuracy". Retrieved 2026-11-03.
  9. Apple Dictation on-device processing: Apple support page. Keyboard settings show whether voice input is "processed on your device and not sent to Siri servers". With Apple Intelligence in macOS 27, dictation uses an on-device model to improve spelling, punctuation, and capitalization, in English on Macs with M3 or later and at least 12 GB of unified memory. Retrieved 2026-11-03.
  10. Evoglyph cleanup-stage quality and eval-harness latency (p50 207ms, p95 1019ms on the 117-fixture editorial-cleanup eval set), on the production prompt and LoRA bundle that ship in the app: internal eval results, app-repo eval/results/v5p6-shipping-baseline-2026-05-11.md. Hardware: Apple M4 Max, 64 GB. 2026-05-11.
  11. Wispr Flow's stated latency target, "within 700ms of when they stop speaking", for full transcription and LLM formatting: Wispr Flow, Technical challenges (2025-09-11). A goal Wispr sets for itself, not a measurement. Retrieved 2026-11-03.