Representação de palavras para aplicações de aprendizado profundo no processamento de linguagem natural

Abstract

Natural Language Processing (NLP) has benefited greatly from the use of Deep Learning. For natural language sentences to be processed by neural networks, sentence words must be represented by numeric values. In this work, three systems that support the representation of words in numerical format are analyzed: Word2Vec, ELMo and BERT. A neural network was set up in which the initial layers are mapped from pre-trained models of these systems and the final layers use the word representation to perform the NLP task: sentiment analysis. A study of the involved architectures of the three chosen systems was performed and an analysis of the performance of the network used as reference for comparison of the three systems was made.