Novo logótipo da Solutional (1)

BERT (modelo linguístico)

Partilhar isto
" Voltar ao Índice do Glossário

BERT, short for Bidirectional Encoder Representations from Transformers, is a language model developed by Google[1]. This model uses a method called WordPiece to convert English words into integer codes, and is capable of understanding the context of words in both directions, left and right. BERT comes in two versions, BASE and LARGE, the latter being bigger with 12 transformer encoders. This model, however, doesn’t include a decoder, which makes generating text a bit challenging. BERT has been recognized for its high performance in natural language understanding tasks, even winning an award at the 2019 NAACL Conference. It has been influential in the field of natural language processing, sparking the development of other models. Google uses BERT to enhance its search algorithms, and it’s also used for text classification, machine comprehension, and more. Numerous studies and papers have been published on BERT, contributing to our understanding of its impact and effectiveness.

Definições de termos
1. Google ( Google ) A Google é uma empresa tecnológica reconhecida mundialmente, conhecida principalmente pelo seu motor de pesquisa. Fundada em 1998 por Larry Page e Sergey Brin, a empresa cresceu muito, diversificando-se em vários sectores relacionados com a tecnologia. A Google fornece uma vasta gama de produtos e serviços, incluindo Gmail, Maps, Cloud, YouTube e Android. Também produz hardware como os smartphones Pixel e os Chromebooks. A empresa, agora parte da Alphabet Inc. desde 2015, é conhecida pela sua inovação e cultura no local de trabalho, incentivando os funcionários a trabalhar em projectos pessoais. Apesar de enfrentar várias questões legais e éticas, a Google continua a ter impacto na indústria tecnológica com as suas inovações e avanços técnicos, como o desenvolvimento do sistema operativo Android e a aquisição de empresas centradas na IA.

Bidirectional Encoder Representations from Transformers (BERT) is a language model based on the transformer architecture, notable for its dramatic improvement over previous state of the art models. It was introduced in October 2018 by researchers at Google. A 2020 literature survey concluded that "in a little over a year, BERT has become a ubiquitous baseline in Natural Language Processing (NLP) experiments counting over 150 research publications analyzing and improving the model."

BERT was originally implemented in the English language at two model sizes: (1) BERTBASE: 12 encoders with 12 bidirectional self-attention heads totaling 110 million parameters, and (2) BERTLARGE: 24 encoders with 16 bidirectional self-attention heads totaling 340 million parameters. Both models were pre-trained on the Toronto BookCorpus (800M words) and English Wikipedia (2,500M words).

" Voltar ao Índice do Glossário
pt_PT_ao90PT
Deslocar para o topo