search

PolinaZulik / metaphor-psycho

0 stars 0 forks source link

Test Bert-based metaphor identification on Russian wiktionary verbs corpus #7

Closed PolinaZulik closed 2 years ago

PolinaZulik commented 2 years ago

only examples with verb forms and lemmas present; only verbs for which both met and non-met examples are present: 4,740 examples. out of these 1,815 metaphoric (0.383) , 2,925 non-metaphoric (0.617). NLI, no fine-tuning, with 'cointegrated/rubert-base-cased-nli-twoway':

text1 = 'Только они не бомбардируют газеты письмами наравне с откровенными психами , хотя , по правде говоря , не особенно от них отличаются .'
text2 = '\'Бомбардируют\' - это метафора.'
Verb P R F1 Acc
Binary 0.392 0.787 0.524 0.451
Macro-average 0.435 0.795 0.527 0.473
fine-tuned up to 2 epochs: Verb P R F1 Acc
Binary 0.644 0.502 0.562 0.701
PolinaZulik commented 2 years ago
fine-tuned up to 2 epochs: Verb P R F1 Acc
Binary 0.644 0.502 0.562 0.701
PolinaZulik commented 2 years ago
on train/val/test sample, 25 epochs, 1st epoch was the best in terms of val_loss and f1, so used the first epoch model. stopping on val loss. without quotes: 20 epochs, early stop with patience = 8. also first epoch model was the best. Desing Verb P R F1 Acc
NLI with quotes Binary 0.60 0.71 0.65 0.71
NLI without quotes Binary 0.61 0.69 0.64 0.71