But few people knew the language well enough to manually transcribe the audio. Inspired by voice assistants like Siri, Mahelona began to explore the possibilities of natural language processing. “Teach the computer to speak Maori has become absolutely essential,” says Jones.
But Te Hiku was faced with a chicken-and-egg problem. To build te reo The speech recognition model required an abundance of transcribed audio. In order to transcribe sound, he needed advanced speakers, the small amount of which he tried to make up for in the first place. However, there were many beginning and intermediate speakers who could read te reo the words out loud better than they could recognize them on the record.
So Jones and Mahelona, along with Te Hiku COO Susannah Duncan, came up with a clever solution: instead of transcribing an existing sound, they asked people to record as they read a series of sentences designed to capture the full spectrum of the language’s sounds. For the algorithm, the resulting dataset will perform the same function. From these thousand pairs of oral and written sentences, he would have learned to recognize te reo syllables in audi.
The team announced a competition. Jones, Mahelona and Duncan contacted every Maori community they could find, including traditional capa khaka dance groups and waka ama canoe racing teams, and it turned out that whoever submitted the most entries would win the grand prize of $5,000.
The whole society is mobilized. The competition has flared up. Member of the Maori community, Te Mihinga Komene, educator and advocate for digital revitalization te reorecorded only 4,000 phrases.
Money wasn’t the only motivator. People have embraced Te Hiku’s vision and trusted it to protect their data. “Te Hiku Media said: “What you give us, we are here as kaitiaki [guardians]. We take care of that, but you still own your sound,” says Te Mihinga. “It is important. These values define who we are as Maori.”
Within 10 days, Te Hiku collected 310 hours of speech-text pairs from approximately 200,000 entries made by approximately 2,500 people, an unprecedented level of research participation in the AI community. “No one could have done this but a Maori organization,” says Caleb Moses, a Maori data scientist who joined the project after learning about it on social media.
The amount of data was still small compared to the thousands of hours typically used to train English models, but it was enough to get you started. Using data to bootstrap an existing open source model from the Mozilla Foundation, Te Hiku created his very first te reo speech recognition model with 86% accuracy.