/ Bei Wang: Topology of word embeddings

Bei Wang: Topology of word embeddings

April 11, 2023
4:30 pm - 5:30 pm

Abstract: Transformer-based language models such as BERT and its variants have found widespread use in natural language processing. A common way of using these models is to fine-tune them to improve their performance on a specific task. However, it is currently unclear how the fine-tuning process affects the underlying structure of the word embeddings from these models. In this talk, I will discuss recent efforts in exploring the topology of these fine-tuned word embeddings. This is joint work with Archit Rathore, Yichu Zhou, and Vivek Srikumar.