search

GiovanniPioDelvecchio / NLP-Project

This is the repository for the group DP(G)R dedicated to the project for the exam of NLP, which will consist in a model for prediction of values associated to argumentations.
0 stars 0 forks source link

Creation of a BERT-like baseline #5

Closed GiovanniPioDelvecchio closed 1 year ago

GiovanniPioDelvecchio commented 1 year ago

It is necessary to extend the baseline by adding a language model at its base, in particular one can choose between the following transformer-based language models:

DomMcOyle commented 1 year ago

paper mentioned in the dataset one. Allows learning rules capturing relations between labels: https://link.springer.com/article/10.1007/s10994-016-5552-1

DomMcOyle commented 1 year ago

the more suitable models for testing are: ELECTRA ConvBERT BORT Funnel Transformers the others could f**k themselves

DomMcOyle commented 1 year ago

BORT tested with the same head, gave 0.24 F1 i think it can be discarded

DomMcOyle commented 1 year ago

Funnel gave 0.3765 F1 but probably it compresses the model too much. Probably one could try to finetune it directly.