Closed AtillaKaanAlkan closed 1 year ago
you annotated the data using LingMessCoref
. Then CorefTrainer
create another model of FCorefModel
(this is the whole idea of distillation) which is not optimized (yet) for coreference task.
Thanks a lot for your answer, it helps me to understand the point I was missing! I got the expected results now! I can close the issue :)
Hi @shon-otmazgin !
First of all, thanks for this package !
I have followed the instructions in the « Distil your own coref model » section in order to produce annotations of my corpus.
Here are the steps I followed :
It then generated a jsonlines-format file with the following keys for each document in the file: [‘text’, ‘clusters’, ‘clusters_strings’]
Secondly, I give to the trainer its own predictions (the output file generated at the first step) for evaluation purpose :
I have not done any fine-tuning so far (so I did not run the
trainer.train()
command), I just give to the model its own predictions, and thus, I was expecting all the metrics (Precision, Recall and F1 score) to be 100 % because I am comparing two files/predictions which are the same. However, I obtained a score of 0 for each metrics. I am not understanding that point. How is this possible to get this null score ? Am I missing something ?Thanks a lot for your help!
Best, Atilla