African Language Speech Data: How to Collect and Prepare it

Write a transcription style guide for each language before production starts, then test it on a pilot batch. Without shared rules, two transcribers produce two different datasets.

Build the guide in this order:

1. Fix the orthography. Name the spelling standard. Decide how to handle Yoruba tone marks and Hausa hooked letters, and whether missing diacritics count as errors.
2. Fix the script rules. Amharic uses the Ethiopic script. The Unicode Standard models its core block as 43 consonants crossed with 8 vowels, or 344 syllables. It also notes that the traditional Ethiopic wordspace is giving way to a plain space in modern use. Pick one and enforce it.
3. Define code-switching tags. Keep English or French words in their own spelling and tag them, so the model learns real speech patterns.
4. Set number and date rules. For speech recognition, transcribe numbers as spoken words unless your pipeline normalizes them later.
5. List non-speech tags. Standardize tags such as [noise], [laugh] and [unclear], plus a rule for hesitations and false starts.
6. Normalize the text. Save everything as UTF-8 in one Unicode normalization form so identical words match.
7. Pilot and revise. Have two native linguists transcribe the same clips, compare results and update the guide where they disagree.