Learning from Data (Fall 2022)

[kundtx]

http://8.129.175.102/lfd2022fall-poster-session/1.html

[min108]

G25 Min Jie: Great work! I notice that your best results were performed on the model BERT, could you state some advantages of this model?

[Prof-Greatfellow]

@min108 G25 Min Jie: Great work! I notice that your best results were performed on the model BERT, could you state some advantages of this model?

G1 Haizhou Liu: Thank you very much for your recognition of our work. 1) The main advantage of BERT lies in the transformer architecture, which adopts multi-headed attention to learn the importance of contexts w.r.t. the current sentence/token. Due to a much more complicated architecture, BERT is expected to learn both short-term and long-term language dependencies more accurately. 2) Also, BERT encodes both token embeddings and positional encodings, which means that there are more "features" for training. 3) Finally, BERT is pre-trained on a much more gigantic dataset (e.g., the Wikiepdia corpus) before being fine-tuned to our own dataset, so that it has a much better understanding of natural language than models trained from scratch. All the above leads to a higher prediction accuracy of BERT models. Thanks again for taking an interest in our project and the BERT model.