Research· Benchmark · November 2026

STORM ASR 2.0 benchmark

Our most accurate speech recognition for Nigerian languages yet, tested against the leading Nigerian speech recognition models on the same 22,921 recordings in Yoruba, Hausa, Igbo, Nigerian Pidgin, Nigerian English and code-mixed speech.

words wrong per 100 across all languages, the lowest of every model tested
15.6
fewer word errors than the next best model (22.6)
31%
test groups (five languages and code-mixed) where STORM ASR 2.0 is most accurate
6 of 6
letters wrong per 100, with tone marks counted
5.6

Words wrong per 100 words spoken

Lower is better. Same 22,921 recordings for every model.

  • STORM ASR 2.0
  • SBPN Large
  • Meta MMS-1B
English
4.5
9.0
19.3
Hausa
11.9
18.1
28.2
Igbo
22.2
28.7
52.3
Pidgin
14.2
15.5
50.4
Yoruba
19.4
24.5
51.6
Code-mixed
14.8
25.1
69.0
All languages
15.6
22.6
51.1

Word error rate

Words wrong per 100 words spoken. A wrong, missing or extra word each count as one error. Best result in each row highlighted.

LanguageSTORM ASR 2.0SBPN LargeMeta MMS-1BNCAIR
Nigerian English4.59.019.312.1
Hausa11.918.128.2n/a
Igbo22.228.752.3n/a
Nigerian Pidgin14.215.550.4n/a
Yoruba19.424.551.644.3
Code-mixed (Yoruba + English)14.825.169.058.0
All languages15.622.651.1n/a

Letter error rate

Letters wrong per 100 letters. A missing or wrong tone mark counts as a wrong letter, so this shows how well each model writes Yoruba, Igbo and Hausa as they are really written.

LanguageSTORM ASR 2.0SBPN LargeMeta MMS-1BNCAIR
Nigerian English1.93.56.86.1
Hausa3.24.77.4n/a
Igbo8.010.117.6n/a
Nigerian Pidgin7.47.621.4n/a
Yoruba7.79.818.820.1
Code-mixed (Yoruba + English)5.311.127.436.4
All languages5.68.718.7n/a

Examples

Recordings from the test set, as each model wrote them.

Code-mixed
What was saidBabangida ló hand over power fún interim government
STORM ASR 2.0Babangida ló hand over power fún interim government
SBPN Largebabangida ló wan overpower fún interim government
Yoruba
What was saidÀlùfáà ìjọ aṣọ ara kan ṣe ìṣirò ìgbà tí Jésù yóò dé.
STORM ASR 2.0Àlùfáà ìjọ aṣọ ara kan ṣe ìṣirò ìgbà tí Jésù yóò dé.
SBPN Largeàlùfáà ìjọ aṣọ ara kan ṣe ìṣe èrò tí jésù yóò dé
Igbo
What was saida naghị ekwe ka nwaanyị di ya nwụrụ rie nri e siiri ndị ezinụlọ ya
STORM ASR 2.0A naghị ekwe ka nwaanyị di ya nwụrụ rie nri e siiri ndị ezinụlọ ya
SBPN Largeanaghị ekwe ka nwaanyị di ya nwụrụ rie nri e siri ndị ezinụlọ ya
Nigerian Pidgin
What was saidChicken wey run way from Borno go Ibadan go still end up inside pot of soup
STORM ASR 2.0Chicken wey run way from Borno go Ibadan go still end up inside pot of soup
SBPN Largechicken wey run away from borno go ibadan go still end up inside port of soup
Nigerian English
What was saidFirst, Hurricane Rosa's path leads to landfall across the Baja California peninsula of Mexico.
STORM ASR 2.0First, Hurricane Rosa's path leads to landfall across the Baja California peninsula of Mexico.
SBPN Largefirst hurricane rose's path leads to landfall across the baha california peninsula of mexico

How we tested

  • Test set. 22,921 recordings, about 34 hours, in Nigerian English, Hausa, Igbo, Nigerian Pidgin, Yoruba and Yoruba mixed with English. Every model transcribed exactly the same recordings.
  • STORM ASR 2.0 was measured through the live production service, exactly as customers receive it, with no language given: it recognises the language on its own. Meta MMS-1B and the NCAIR models were given the correct language for every recording.
  • Scoring. Capitals and punctuation are ignored; tone marks count, so a missing or wrong tone mark makes a word wrong. Recordings whose reference text contains digits were left out, because models write numbers differently. All counts were made by our own scoring code from each model's raw transcripts.

Models compared

  • SBPN Large (ogunlao/SBPN_multilingual_large), a multilingual model for Nigerian languages, licensed for research use only.
  • Meta MMS-1B (facebook/mms-1b-all) with its Yoruba, Hausa, Igbo, Pidgin and English adapters.
  • NCAIR Yoruba-ASR and NigerianAccentedEnglish, which cover Yoruba and Nigerian English only.

Each was run from its published weights with its default settings.

STORM ASR 2.0 is available now.

Files and real-time streams, no language setting needed, one API key.