search

ljynlp / W2NER

Source code for AAAI 2022 paper: Unified Named Entity Recognition as Word-Word Relation Classification
MIT License
510 stars 81 forks source link

Is an ner (ground truth) required for testing? #75

Open domyown opened 2 years ago

domyown commented 2 years ago

Hello, first of all, thank you for giving me insights in the field of information extraction through your paper. I have a question regarding the test data. Each data consists of a dictionary with sentence and ner keys, and ner is identified as the ground truth entities that exist in the sentence. Therefore, when testing the model with the unseen data (when there is only a test text = when there is only a sentence value), i think it doesn't matter if the ner value is an empty list. (=entity text is also an emty set) However, when ner value is an empty list, it seems that the outputs of the model cannot be decoded. Could you give me some comments or feedback on this issue?

The code regarding the issue is as follows. ent_c, ent_p, ent_r, decode_entities = utils.decode(outputs.cpu().numpy(), entity_text, length.cpu().numpy())

ljynlp commented 2 years ago

Hi, our code works when the ner value is an empty list in the test set, although the f1 will be 0. I have tested it on the resume dataset, and I got the correct output.json file. Do you mean your process is blocked by the decode function?