search

Hironsan / anago

Bidirectional LSTM-CRF and ELMo for Named-Entity Recognition, Part-of-Speech Tagging and so on.
https://anago.herokuapp.com/
MIT License
1.48k stars 371 forks source link

F1-Score on Test Set #56

Closed slouvan closed 6 years ago

slouvan commented 6 years ago

Hi @Hironsan , I just want to clarify, for the performance in the test set, is the F-1 score that you obtained is indeed 90.67? I tried your code but I can only obtain around 86 of F1-score.

Then, I tried to download the pretrained model and then call model.evaluate on the test set, and it yields 98 of F1 score on the test set which I found does not make sense. In Lample's paper, the performance is around 90 of F1. Can you comment on this?

Thanks

Hironsan commented 6 years ago

Thank you for your question. I replied to this question at https://github.com/Hironsan/anago/issues/66 .

with best regards.

nkrot commented 4 years ago

@slouvan , you are lucky if you managed to obtain F1 score of 86. I ran training of anago on train+valid dataset with different number of epochs (20,30,40,50,60, etc) and the best i have achieved was around 82 (with 20 and 70 epochs). @Hironsan , what am i doing wrong? Can it be the case that the champion score of 90.67 was obtained with a different set of hyperparameters?