Meta introduces language model MMS that is “bigger than ChatGPT”


Meta has developed its personal language mannequin. Massively Multilingual Speech (MMS) just isn’t a clone of OpenAI’s ChatGPT for as soon as.

MMS can acknowledge greater than 4,000 spoken languages ​​and helps text-to-speech for 1,100 languages. As is custom, Meta is making its initiatives open supply, and now MMS can also be open supply, “to protect language range and encourage researchers to construct on that basis,” the social media platform mentioned. know.

Growing speech recognition and text-to-speech fashions sometimes requires 1000’s of hours of audio coaching with related transcription tags. The latter are essential for the algorithms to appropriately categorize and perceive knowledge. Within the case of languages ​​that aren’t (a lot) utilized in trendy society, the language mannequin is usually a means to forestall that wealth from disappearing.

MMS makes use of spiritual texts

It’s putting that Meta took an uncommon method to accumulating the audio knowledge. For instance, it was based mostly on recordings of translated spiritual texts. “We used spiritual texts, equivalent to these within the Bible, which have been translated into many languages ​​through the years and whose translations have already been extensively studied for text-based translation analysis,” mentioned Zuckerberg and co. On this manner, the researchers would have succeeded in rising the out there languages ​​for the mannequin to greater than 4,000.

“Though the content material of the recordings is spiritual, our analysis reveals that this doesn’t bias the manufacturing of much more spiritual language,” Meta wrote. “That is as a result of our method is predicated on a ‘connectionist temporal classification’ (CTC), which is rather more compact and centered than different massive language fashions (LLMs). As well as, each women and men have recorded textual content,” it sounds.

Subsequently, Meta began working with its wav2vec 2.0, a self-learning mannequin that may practice based mostly on unlabeled knowledge. “The outcomes are good. They present that the Massively Multilingual Speech mannequin performs very properly in comparison with current fashions. It helps 11 occasions as many languages ​​as OpenAI’s Whisper,” the researchers conclude.