Quran verses in the khutbah
Right when shown
98%
233 of 237 verses shown were the right verse
Found
about 94% of the verses quoted
Measured accuracy
Public test recordings, run through the moschee.io live translation and scored by the rules Baian describes for its own figures. Our figure is ahead of Baian's on all 6 language pairs we could compare, and the finished line appears sooner. Each lead is within the margin of error, and our weak spots are on this page too.
6 of 6
language pairs ahead of the figure Baian publishes
by 4.9 points on average, each lead within the margin of error
1.8 s
from the last word to the finished translated line (median)
Baian publishes 2.1 s
98%
of the Quran verses shown during the khutbah were the right verse
233 of 237, first measurement on 17 live sermons
Translation quality
The test works like this: a speaker reads a sentence aloud, the live translation hears it and translates it. An automatic judge, MetricX-24, then rates our translation of the speech and a professional translation of the written sentence.
The figure is the share of sentences where the judge rates ours higher; a tie counts half. 50% means level with the professional translation. Baian publishes the same figure for its own product, which is what makes a comparison possible.
The language pairs Baian publishes
English → Arabic
moschee.io, measured:63.7%95% range 58.6–69.0%
Baian, published:58.0%
English → Hindi
moschee.io, measured:70.3%95% range 65.6–75.0%
Baian, published:65.5%
English → Korean
moschee.io, measured:68.3%95% range 63.3–73.0%
Baian, published:61.5%
English → Hausa
moschee.io, measured:76.9%95% range 72.4–81.6%
Baian, published:75.0%
English → Swahili
moschee.io, measured:71.1%95% range 66.6–75.7%
Baian, published:68.0%
English → Urdu
moschee.io, measured:74.1%95% range 69.7–79.1%
Baian, published:67.0%
This is how an Arabic khutbah is translated. Baian publishes no translation figures for this direction.
Arabic → German
moschee.io, measured:53.0%95% range 47.2–58.8%
Arabic → English
moschee.io, measured:62.4%95% range 56.4–67.7%
Arabic → Turkish
moschee.io, measured:62.2%95% range 56.5–67.7%
Arabic → French
moschee.io, measured:61.5%95% range 55.8–67.0%
Arabic → Bosnian
moschee.io, measured:57.4%95% range 51.9–62.7%
Arabic → Albanian
moschee.io, measured:59.7%95% range 54.1–65.7%
Baian publishes no ranges. The ones shown for Baian are our estimate from its 148 sentences.
| Language pair | Sentences | moschee.io | Against the professional translation | Baian (published) | Lead in points |
|---|---|---|---|---|---|
| English → Arabic | Sentences350 | moschee.io63.7%95% range 58.6–69.0% | Against the professional translationRated higher | Baian (published)58.0%Live product | Lead in points+5.7 |
| English → Hindi | Sentences350 | moschee.io70.3%95% range 65.6–75.0% | Against the professional translationRated higher | Baian (published)65.5%Live product | Lead in points+4.8 |
| English → Korean | Sentences350 | moschee.io68.3%95% range 63.3–73.0% | Against the professional translationRated higher | Baian (published)61.5%Live product | Lead in points+6.8 |
| English → Hausa | Sentences350 | moschee.io76.9%95% range 72.4–81.6% | Against the professional translationRated higher | Baian (published)75.0%Caption tool for recordings | Lead in points+1.9 |
| English → Swahili | Sentences350 | moschee.io71.1%95% range 66.6–75.7% | Against the professional translationRated higher | Baian (published)68.0%Caption tool for recordings | Lead in points+3.1 |
| English → Urdu | Sentences350 | moschee.io74.1%95% range 69.7–79.1% | Against the professional translationRated higher | Baian (published)67.0%Caption tool for recordings | Lead in points+7.1 |
| Arabic → German | Sentences283 | moschee.io53.0%95% range 47.2–58.8% | Against the professional translationLevel | Baian (published)Not published | Lead in pointsNot published |
| Arabic → English | Sentences283 | moschee.io62.4%95% range 56.4–67.7% | Against the professional translationLevel | Baian (published)Not published | Lead in pointsNot published |
| Arabic → Turkish | Sentences283 | moschee.io62.2%95% range 56.5–67.7% | Against the professional translationLevel | Baian (published)Not published | Lead in pointsNot published |
| Arabic → French | Sentences283 | moschee.io61.5%95% range 55.8–67.0% | Against the professional translationLevel | Baian (published)Not published | Lead in pointsNot published |
| Arabic → Bosnian | Sentences283 | moschee.io57.4%95% range 51.9–62.7% | Against the professional translationLevel | Baian (published)Not published | Lead in pointsNot published |
| Arabic → Albanian | Sentences283 | moschee.io59.7%95% range 54.1–65.7% | Against the professional translationLevel | Baian (published)Not published | Lead in pointsNot published |
Delay
We measure from the last voiced moment of a sentence to the moment the finished translated line is on screen. The median is the value half of the sentences stay under; nine in ten sentences stay under the second value.
Arabic speech
402 recordings
Median 1.80 s
Nine in ten sentences within 2.92 s
English speech
601 recordings
Median 1.83 s
Nine in ten sentences within 3.11 s
Baian (published)
Median 2.1 s
Speech recognition
Before anything can be translated, the speech has to be written down correctly. The word error rate (WER) is the share of words that are wrong, missing or added; for Korean, characters are counted instead (CER). Lower is better.
Our figures use the text cleaning Baian uses for its own: punctuation, capitalization, Arabic vowel marks and the way numbers are written are ignored.
| Language | Recordings | Measure | moschee.io | Baian (published) | Lead in points | Standing |
|---|---|---|---|---|---|---|
| Arabic | Recordings428 | MeasureWER | moschee.io5.76%95% range 4.71–6.95% | Baian (published)5.1%Live product | Lead in points−0.7 | StandingLevel |
| English | Recordings647 | MeasureWER | moschee.io5.06%95% range 4.55–5.61% | Baian (published)Not published | Lead in pointsNot published | StandingNo Baian figure |
| German | Recordings200 | MeasureWER | moschee.io5.98%95% range 4.75–7.25% | Baian (published)Not published | Lead in pointsNot published | StandingNo Baian figure |
| Turkish | Recordings200 | MeasureWER | moschee.io5.99%95% range 4.79–7.14% | Baian (published)Not published | Lead in pointsNot published | StandingNo Baian figure |
| French | Recordings200 | MeasureWER | moschee.io5.14%95% range 4.16–6.15% | Baian (published)4.3%Caption tool for recordings | Lead in points−0.8 | StandingLevel |
| Russian | Recordings200 | MeasureWER | moschee.io5.61%95% range 4.59–6.71% | Baian (published)6.4%Live product4.7%Caption tool for recordings | Lead in points+0.8Live product−0.9Caption tool for recordings | StandingLevel |
| Danish | Recordings200 | MeasureWER | moschee.io9.14%95% range 7.65–10.68% | Baian (published)9.9%Live product | Lead in points+0.8 | StandingLevel |
| Korean | Recordings200 | MeasureCER | moschee.io5.27%95% range 3.95–6.82% | Baian (published)2.9%Live product | Lead in points−2.4 | StandingBehind |
| Urdu | Recordings200 | MeasureWER | moschee.io15.86%Stricter cleaning · 95% range 14.15–17.51%13.61%Baian's cleaning · 95% range 12.30–14.81% | Baian (published)13.2%Caption tool for recordings · Stricter cleaning | Lead in points−2.7Stricter cleaning | StandingBehind |
Quran and hadith
First measurement, November 2026
When the imam quotes a Quran verse or a hadith, moschee.io shows listeners its source. For a verse, a published translation appears instead of a machine translation, where one exists for the language.
We went through 17 live Friday sermons held in the 90 days before the measurement and checked every source that listeners were shown against what was said. Baian publishes no figure for this, so there is nothing to compare with here.
Right when shown
98%
233 of 237 verses shown were the right verse
Found
about 94% of the verses quoted
Right when shown
97%
374 of 387 verses shown were the right verse
Found
Not measured yet
Right when shown
85%
98 of 115 hadith shown were the right hadith
Found
about 68% of the hadith quoted
Method and limits
Comparing two products is only fair if the reader can check it. This is everything that went into the figures above.
Speech: the test split of FLEURS, a public collection of read sentences, with 428 recordings in Egyptian Arabic and 647 in US English; for the other speech recognition languages, 200 recordings each. For translation, one recording per sentence: 350 English and 283 Arabic sentences. Translations to compare with: the professional translations of the same sentences in FLoRes-200.
Each recording went through the moschee.io live translation pipeline at real speed, one sentence per stream, with the spoken language set in advance. This is how AssemblyAI ran the engines Baian compares itself with. Baian streamed 15-minute recordings with automatic language detection, which is the more demanding setup.
MetricX-24 in its reference-free mode rates each translation on its own. We count how often our translation of the speech scores better than the professional translation of the text; a tie counts half. An automatic judge is not a human reader, and such judges can favor machine translation. A share above 50% therefore does not prove that a translation is better than a professional's. The figure is suited to comparing two systems that are scored the same way.
Word error rate under Baian's text cleaning, so that the figures can be compared. Internally we also score with stricter cleanings, which give higher word error rates.
Our 95% ranges come from bootstrap resampling of the sentences. Baian publishes no ranges. With 148 sentences, each of its figures carries about ±8 points; ours carry about ±5. For the difference between the two that gives about ±9 points. Each lead lies inside that margin. On average we are 4.9 points ahead.
From the last voiced audio frame of a sentence to the finished translated line on screen, on the recordings in which the recognizer closed the sentence on its own.
Copied from baian.ai/accuracy as we read it on 8 November 2026. We did not test Baian's product ourselves, and the sentences are not identical: Baian used 148 English sentences, we used 350. If Baian has updated its page since, the figures there apply.
FLEURS is read speech recorded in quiet surroundings. A Friday sermon has reverberation, emotion, dialect and Quran recitation, so accuracy in a real sermon can be lower for every system. It is also a single run: a live translation does not word a sentence the same way twice, and between two runs on the day of the measurement a pair moved by about 2 points.
It is the share of test sentences for which the automatic judge MetricX-24 rated our live translation of the spoken sentence above a professional translation of the written sentence. A tie counts half. 50% means level with the professional translation.
On the 6 language pairs we could compare, our measured figure is higher than the one Baian publishes, by 4.9 points on average, and the finished line appears after 1.8 s against 2.1 s. Each lead is within the margin of error, and the two measurements did not use identical sentences. So we say “ahead on every pair we compared”, not “proven better in every language”. In speech recognition we are level with Baian in 4 languages (Arabic, French, Russian, and Danish) and behind in 2 (Korean and Urdu).
In speech recognition, in 2 languages (Korean and Urdu): for Korean our character error rate is 5.27% against Baian's 2.9%; for Urdu our word error rate is 15.86% against 13.2% under the same rule. For Arabic and French our error rate is higher than Baian's too, but Baian's figure lies inside our 95% range, so we count them as level. Hadith are also recognized less reliably than Quran verses: 85% of the hadith shown were the right hadith, and about 68% of the hadith quoted were found.
Because public test sets are the only way to compare with figures another provider publishes: anyone can obtain the same recordings and check the result. Real sermons are harder for every system. That is why the Quran and hadith figures on this page come from real live sermons.
In 17 live Friday sermons, 98% of the Quran verses shown during the khutbah were the right verse, and about 94% of the verses quoted were found. For hadith the figures are 85% and about 68%. These are first figures; a second check of a sample is still pending.
They were measured on 10 November 2026. We repeat the measurement when the live translation changes, and the date is shown at the top of the page. Baian's figures are quoted as we read them on 8 November 2026.
moschee.io vs Baian
Specialized AI translation for khutbahs, talks, and video captions — without mosque management.
Read comparisonLivestreaming and live translation
How the khutbah reaches listeners as captions and audio, in the prayer room and at home.
Explore the featureAI Khutbah Translation: The Complete Guide for Mosques
How mosques translate the khutbah live with AI: the tech explained, AI vs. human interpreters, how to choose a solution, GDPR, and a respectful rollout.
Read guideStart free with prayer times and the website widget. The website, app content, and AI translation come with the Basis plan, which you can try free for 14 days — no contract, GDPR-compliant, hosted in Europe.