Qwant’s research teams are taking another step forward in applying science to improving everyday life with the publication of a new BERT model.
Have you heard of NLP?
Natural Language Processing is an applied multidisciplinary field of research aimed at developing natural language processing tools capable of responding to a wide range of applications.
Among these, there are a large number of applications useful in everyday life and well known to the general public: speech recognition, text comprehension, translation, thematic classification of documents, and many others.
The challenge (and beauty) of NLP is to make machines understand human language in order to allow them to interact with us. If you want to know more about this technology and the work carried out by Qwant’s teams, we invite you to read this article which followed the Coling conference on the subject.
With the publication by Oralie Cattan, PhD student, and Christophe Servan, scientific manager at Qwant and other researchers of a research article on the usability of BERT models for French, Qwant’s research teams are taking another step forward in the application of science to improving everyday life. These compact, less data-intensive models had previously been offered mainly in English.
But that’s not all! Oralie and Christophe also present a new compact BERT model for French, developed within Qwant, called “FrALBERT” and available on HuggingFace. This is a performance, since this compact model, using “only” 12 million parameters, is one of the few compact BERT model applicable to the French language to exist. In addition, it was driven thanks to the computing power that Qwant has internally. The two main French BERT models existing until then are larger (+ 110 million parameters!), and required the use of a supercomputer (e.g. Jean Zay or through a partnership with a GAFAM).
The Qwant teams unite to congratulate Oralie Cattan and Christophe Servan and reaffirm their commitment to the values of progress through science and knowledge.
For more information: scientific publication: “On the Usability of Transformers-based models for a French Question-Answering task” / HuggingFace
