Kriti Telephony

NepTel benchmark — real Nepali call-center audio

State of the art on Nepali phone speech.

NepTel is the public benchmark for real Nepali telephony: three recorded call-center calls, scored with the official evaluation suite. Kriti Telephony achieves WER 32.38, beating the previous best narrowband system by 1.4 points. The lead is directional on a small corpus — paired bootstrap confirms the direction — and we publish the exact evaluation command so anyone can verify it.

NepTel public benchmark (3 calls, single vendor). Lower is better. General Nepali WER measured on the Kriti V1 dev set.

Why telephony is hard

Phone audio is a different domain.

Phone calls are sampled at 8 kHz, compressed by codecs, and carry background noise, cross-talk, and strong accents. A model trained on clean studio audio degrades sharply when the input arrives over a phone line. Kriti V1 reaches WER 40.51 on NepTel despite its strong general score — the domain gap is real.

Method

Knowledge distillation, not narrow fine-tuning.

Rather than fine-tuning on in-domain data only, we used a strong teacher model to re-label roughly 12,000 clips of conversational Nepali — including code-switched and spontaneous speech from Bijay13 and Shyyamsh — in the target reference style, then mixed in a large clean Nepali anchor to prevent catastrophic forgetting. No NepTel evaluation audio was used during training.

Use cases

Built for real Nepali call infrastructure.

Call-center transcription, IVR voice navigation, phone-based survey collection, helpline analytics, and any pipeline where audio arrives over PSTN or VoIP. Code-switched Nepali — the kind spoken in actual call-centers, mixing Nepali and English mid-sentence — is a first-class target.

Model specifications

Same architecture, adapted domain.

Reproducibility

Every number is checkable.

The evaluation command, scorer version, and exact benchmark split are published in the repository. The base model lineage traces back to AI4Bharat IndicConformer (commit cd09ba7), with multilingual heads pruned and a Nepali-only decoding graph. See also kriti-reproducibility for the verifiable evidence framework shared across the Kriti family.