search

lavis-nlp / spert

PyTorch code for SpERT: Span-based Entity and Relation Transformer
MIT License
692 stars 148 forks source link

Question about ADE dataset #15

Closed zgq7799 closed 4 years ago

zgq7799 commented 4 years ago

Hi, What should I do to get the overlapping and non-overlapping results of the ADE dataset?

markus-eberts commented 4 years ago

Hi, the option to get the non-overlapping results was not included in the GitHub code. I added it in commit 7ca62786d405a380a4df7984a47abe145e50a94a. You can now set 'no_overlapping=True' in your config file. Note that it is not clear to us how previous (mainly BILOU-based) approaches handled overlapping entities for NER evaluation. We remove these entities from the ground truth as well as the predictions to compute the NER metrics.

To reproduce the results of our paper, you should train/test the model on every split and average the results.

markus-eberts commented 4 years ago

Please leave a comment if the issue is still unsolved.