Arabic Speech-to-Text in 2026: 12 Models Tested on Real Gulf Voice Orders, Including Voxtral 3

TL;DR: We ran 9 speech-to-text models on 54 real WhatsApp voice orders from a UAE grocery-delivery service. The biggest finding was not about models. These customers do not only speak Gulf Arabic and English: four of the 18 voice-only orders were partly or fully in Urdu. Models told to expect Arabic could not read those orders at all. Google's Gemini 3.8 Flash recovered 84% of the products customers ordered. The best models you can run on your own hardware, Voxtral Mini 4B Realtime Arabic from Mistral and Whisper large-v3 left to detect the language itself, reached 71%. The model at the top of the public Arabic leaderboard scored 52%.

Update, 21 October 2026: Voxtral 3. Mistral released Voxtral 3 on 20 October, and we ran its three transcription models on the same scored orders the next day, through Mistral's EU-hosted API with zero data retention enabled on our organisation. Voxtral Small Transcribe 3 and Voxtral Mini Transcribe 3 each recovered 23 of 31 products (74%), second only to Gemini 3.8 Flash, and the streaming Voxtral Mini Transcribe Realtime 3 recovered 21 (68%). Details in Finding 6.

---

What we tested

The audio. 54 WhatsApp voice notes, about six minutes in total, from a UAE grocery-delivery service. Customers speak Gulf Arabic, often with a South Asian accent, drop in English brand names ("two box cigarette Davidoff Gold", "Kenza diet", "chicken fresh"), and, as it turned out, sometimes speak Urdu for the whole message.

The answer key. For each order we know which products were delivered. Eighteen orders were placed only by voice, so everything the shop needed came from 35 voice notes. Those 18 orders, 31 products in total, are the scored set.

The question. Not word error rate. For a shop the question is: from this transcript, can you tell what the customer wants? A product counts as recovered only if its name, or an unambiguous brand, appears in the transcript, in Arabic, English or Urdu. "Manchester" counts as cigarettes, because it is a cigarette brand sold in the UAE. چاول counts as rice. "Car" (سيارة) misheard for "cigarette" (سيجارة) does not count, and neither does a fluent sentence the model invented.

The models. We picked the newest models with Arabic support as of October 2026:

• Run on our own hardware (one Apple M4 Pro machine; the audio never left it): Cohere Transcribe Arabic (July 2026), Mistral Voxtral Mini 4B Realtime Arabic (released 8 October 2026), Audar-ASR-V1-Turbo (top of the Open Universal Arabic ASR Leaderboard), lemura-arabic-asr-qwen3 and qwen3-asr-arabic-uae (two Gulf-focused models built on Qwen3-ASR), and OpenAI Whisper large-v3 as the baseline.
• Run through an API: Google Gemini 3.8 Flash, Google Gemini 3.1 Pro and Mistral Voxtral Small 24B, through OpenRouter with zero data retention enforced on every request.
• Added on 21 October 2026: Mistral's Voxtral Small Transcribe 3, Voxtral Mini Transcribe 3 and Voxtral Mini Transcribe Realtime 3 (streamed note by note, as a live caller would), through Mistral's EU-hosted API, with zero data retention enabled on our Mistral organisation. No language was set and no prompt was given. Two other candidates, OpenAI gpt-audio and Qwen3.8 Omni Flash, had no provider offering zero retention, so we left them out rather than send client audio without it.

Where a model lets you choose, we ran it twice: once told the audio is Arabic, once left to detect the language.

---

The results

Audar and lemura scored the same or slightly worse when left to detect the language, so the table shows their Arabic runs. API speed is not comparable to a local run, so we leave it out. For scale, Mistral's API returned all 35 scored notes in 24 seconds with Voxtral Small Transcribe 3 and 15 seconds with Voxtral Mini Transcribe 3, about 3.5 minutes of audio.

Finding 1: Dubai orders groceries in three languages

In February we concluded that telling Whisper the audio is Arabic was the single most useful setting. On this set that is wrong. Four of the 18 voice-only orders were partly or fully in Urdu, and every model fixed to Arabic turned them into nonsense. Left to detect the language, Whisper wrote them out properly:

ایک آہ وائٹ ڈال چھوٹا والا … دو ٹومٹر دو پیاس (one white dal, the small one … two tomatoes, two onions): Whisper large-v3, language detected

چاول دیو چاول tomato (give rice, rice, tomato): Gemini 3.8 Flash

The same setting that cost Whisper nothing on the Arabic orders took it from 16 products to 22. Cohere Transcribe Arabic has no Urdu and no language detection, so these orders are out of its reach whatever you do.

Finding 2: the leaderboard did not predict the shop floor

Audar-ASR-V1-Turbo leads the public Arabic leaderboard and beats Cohere there. On these voice notes it recovered 16 products to Cohere's 17. The leaderboard's test sets are clean recordings in a single language. A WhatsApp voice note from a Dubai grocery customer is neither. If you are choosing a model for real customer audio, test it on real customer audio.

Finding 3: the newest Arabic model is the best one you can self-host

Mistral released Voxtral Mini 4B Realtime Arabic three days before this test, under Apache 2.0, trained specifically on Arabic code-switching. It tied Whisper for the best self-hosted score without needing any language setting, never got stuck in a repetition loop, and caught things the Arabic-only models missed: "كيلو بطاطا، كيلو بصل" (a kilo of potatoes, a kilo of onions), "طوماطو" (tomato) in an Urdu order. It is designed for streaming, so it can transcribe while the customer is still talking.

Finding 4: fluent nonsense is the dangerous failure

Most models fail visibly: a repeated phrase, a garbled word. One model failed invisibly. Given a voice note ordering lentils, tomatoes and onions in Urdu, qwen3-asr-arabic-uae produced a clean, natural Gulf Arabic sentence about making karak tea and bringing coffee. Nothing about it looks wrong. A loop can be caught automatically; a confident invention cannot, which is why the last step below matters.

Finding 5: Gemini is the most accurate, at the cost of sending the audio out

Gemini 3.8 Flash recovered 26 of 31 products and fully captured 15 of 18 orders. It detected Urdu, kept English brand names in Latin script ("Manchester", "KitKat", "Mai Dubai … water, two carton"), and never looped. It is also an external API: the audio goes to Google, even with zero retention. For many shops that is fine. For a client who cannot let customer audio leave their infrastructure, the answer is Voxtral or Whisper, at 71%.

Gemini 3.1 Pro, Google's larger model, did not do better: 25 of 31 products and 14 of 18 orders, one product behind Flash. It made the same misses plus one of its own, writing "ginger ale" where the customer asked for ginger (جنجبل), which would put the wrong product in the bag. It was also much slower through the API: our first run did not finish in time for publication. For transcribing orders, the bigger model buys nothing.

Finding 6: Voxtral 3 is the best Mistral yet, and writes Urdu in Hindi script

Voxtral Small Transcribe 3 and Voxtral Mini Transcribe 3 each recovered 23 of 31 products: three more than Voxtral Small 24B, one more than the best self-hosted models, three fewer than Gemini 3.8 Flash. What they did well:

• No loops. Whisper got stuck repeating itself on two notes in our re-run; no Voxtral 3 model did.
• The Gulf vocabulary the others dropped. Both caught onion (بصل) and ginger (جنزبيل, جنجبل) in an order where Whisper wrote "مسل" and a truncated "جنجب".
• One of the hardest notes. Voxtral Small Transcribe 3 is the first model in this benchmark to read the 3-second cigarette order: it heard "بلاتيني", Platinum, a cigarette brand.

What to watch:

• Urdu comes out in Hindi script. Where Whisper wrote "دو ٹومٹر" in Urdu script, Voxtral 3 wrote "दो टमाटर" in Devanagari, and sometimes used Gujarati script. The words are right and a language model matching against a catalogue reads either, but a person on a Gulf shop floor may not.
• The streaming model drifts on short notes. Voxtral Mini Transcribe Realtime 3 turned "chawal" (rice) into Spanish ("Saúl, tío") and wrote other short notes in Tamil and Macedonian. It finished two products behind the batch models.
• No public weights. Mistral published no weights for these models: you use them through its API, or self-deploy them through an arrangement with Mistral's sales team. Prices and the full model line-up are in our Voxtral 3 overview. For a client whose audio cannot leave its infrastructure, Voxtral Mini 4B Realtime Arabic remains the self-hosted choice. For one that needs the audio to stay in the EU, Voxtral 3 through Mistral's EU-hosted API is now the strongest option we have tested.

Where every model failed

Two orders defeated the first nine models: a cigarette order in a single 3-second note, and an order for cigarettes and lemons spread over three notes of 2 to 5 seconds. Each model garbled them differently, which points at the audio rather than any one model. Very short notes give a model almost nothing to work with. Voxtral Small Transcribe 3 later got the first one; the second still defeats all twelve.

---

What we would build now

1. Detect the language; never force Arabic. If your customers include South Asian residents of the Gulf, an Arabic-only pipeline silently drops their orders.
2. Pick by where the audio may go. If it can leave your infrastructure, Gemini 3.8 Flash. If it must stay in the EU, Voxtral 3 through Mistral's EU-hosted API. If it cannot leave your infrastructure at all, Voxtral Mini 4B Realtime Arabic, with Whisper large-v3 as a second opinion on hard clips.
3. Detect loops and fall back. Repetition is easy to catch automatically. Re-run the clip on the other model or send it to a person.
4. Match against the catalogue, not the dictionary. Feed the transcript and the shop's product list to a language model, so "Manchester" resolves to a cigarette SKU and چاول to rice.
5. Confirm before you deliver. Reply with the order as understood ("2 x Kenza Diet, 1 x KitKat, correct?"). That one message catches both the garbled transcript and the fluent invention.

Steps 3 to 5 are where most of the reliability comes from. The model mostly decides your accuracy ceiling, your cost and where the audio goes.

---

Limits of this test

• Small sample. 54 clips, 18 scored orders, 31 products. Enough to see the shape, not to rank models one product apart.
• One reviewer, one rule. We judged product recovery with the strict rule above, applied the same way to every model.
• A rebuilt harness for the update. The Voxtral 3 results come from a scoring harness we rebuilt on 21 October. Re-running Whisper large-v3 through it gave the same 22 of 31 products as the original run (11 fully captured orders against 12: Whisper's output varies slightly between runs). Voxtral 3 was run on the 35 scored notes only.
• The answer key is what was delivered, not what was said, so a substitution by the shop would count against every model equally.
• Prompted models got a prompt. Gemini and Voxtral Small were asked to transcribe verbatim, keep Arabic in Arabic script and English in Latin script, and not translate. Voxtral Small sometimes translated anyway.
• Not tested: Falcon-ASR from TII (October 2026, Emirati-focused), which has no published weights or API yet; CNTXT's Munsit and Speechmatics' Arabic-English model, which need an access request or an account; OpenAI gpt-audio and Qwen3.8 Omni Flash, which had no zero-retention provider. We stopped Xiaomi MiMo v2.6 Pro partway: it took hours through the API, and on the 11 voice-only orders it finished it recovered only 2 to 4 of 15 products (depending on how generously you read a garbled brand name), often writing fluent Arabic that had nothing to do with the recording. It is not in the table. We do not send client audio to public demos.

Related: the February benchmark with Whisper variants, a fine-tuned model, ElevenLabs and Hamsa, and how we work with companies in Dubai and the UAE.