workwriting
résumé

Have something worth building?

0xkhingx@gmail.com
Dre · software engineer, Lagos
workwritingrésumégithublinkedinx

© 2026 Oluwadamilare Ogundele

back to top ↑
← Work

Nigerian Language Classifier

Five Nigerian languages, told apart by character texture.

NLP · data to evaluation · 2025 · Python, scikit-learn, NLP, text classification

Source ↗

The text mainstream tools misread

Generic language detectors often misclassify Nigerian languages, so the goal was a focused classifier for English, Yoruba, Igbo, Hausa, and Pidgin that handled local data instead of pretending the problem was already solved. Real-world text made it harder: English drawn from books behaves differently than English drawn from social media, and Pidgin lives mostly in tweets.

A corpus stitched from four worlds

The training set spans 64k+ sentences drawn from NaijaSenti tweets, JW300 parallel text, Wikipedia articles, and Gutenberg books — deliberately mixed registers so the model meets the language where people actually type it.

Why character n-grams, not words

Word-level features would have required per-language vocabulary decisions. Character n-grams, two through five characters and TF-IDF weighted, treat all five languages uniformly and pick up the sub-word signals — morphology, digraphs, diacritics, borrowings — that actually distinguish Yoruba and Igbo without tokenizers that assume spaces mean what they mean in English.

Publishing the error table next to the headline

The final model reached 99.4% accuracy on the held-out test set — and the failure modes are documented beside it. Yoruba with diacritics classifies near 97% confidence but drops toward 89% when users strip diacritics; Pidgin sits near 82% with the least training data; inputs under five words are unreliable; code-switched sentences, utterly normal in Lagos conversation, are not handled at all.

Where it landed

The final model reached 99.4% accuracy on the held-out test set — and I published the error table alongside it. Yoruba with diacritics classifies at ~97% confidence but drops toward 89% when users strip diacritics; Pidgin sits near 82% with the least training data; inputs under five words are unreliable; code-switched sentences, utterly normal in Lagos conversation, are not handled at all.

The headline number was the least interesting part; the error table taught me more about the languages than the training did. Next: more Pidgin data, diacritic-robust training pairs, a FastAPI serving wrapper, and possibly a Hugging Face Space demo.

I wrote about it: classifying nigerian languages

NextMoodMix→