allenai / allennlp

An open-source NLP research library, built on PyTorch.

http://www.allennlp.org

Apache License 2.0

11.76k stars 2.25k forks source link

compute span-f1 with label_encoding :"BIOUL" #3156

Open Maybewuss opened 5 years ago

Maybewuss commented 5 years ago

System (please complete the following information):

OS: Mac os
Python version: 3.6
AllenNLP version: latest
PyTorch version: 1.2

Question

I want to compute span-f1 with 'BIOUL' label encoding, and I find that the function 'bioul_tags_to_spans' does not allow ill-formed spans. But the prediction output of the model may be ill-formed, and will raise 'InvalidTagSequence' Error, so that the span-f1 can not compute.

DeNeutoy commented 5 years ago

Hi,

This is because this code here:

https://github.com/allenai/allennlp/blob/master/allennlp/data/dataset_readers/dataset_utils/span_utils.py#L254

doesn't handle the case of ill formed tag sequences. If you wanted to contribute a fix to allow it to handle ill formed sequences, that would be great!

Maybewuss commented 5 years ago

3169 @DeNeutoy I create a PR to ignore ill-formed spans, thanks !