Is the model able to handle partially annotated training data?

[DanqingZ]

Hi, I am looking into the data (for example conll2003). Although for different entity, we can generate different context, query pair. For each context, we have to generate the (context, question, answer) for all entities. I am wondering if I have partially annotated training data, can I only generate (context, question, answer) when we have the entity in the context?

Thank you!

[ghost]

Thanks for asking! You should add an IF-Statementif not tmp_impossible: before this line https://github.com/ShannonAI/mrc-for-flat-nested-ner/blob/master/data_preprocess/generate_mrc_dataset.py#L91. After that, run script/data/gen_mrc_ner_datasets.sh and only (context, question, answer) pairs with entities in the context will be saved.
tmp_impossible is True denotes that no entities are in the context. Otherwise, at least one entity with the entity type query exists in the context.
I hope this clarifies your question.

[DanqingZ]

thank you for answering my question. I am actually confused about the following questions:

• if I remove all the negative examples for one label. say, only include the (context, question, answer) when query exists in the context, will this degrade model performance?
• when I have partially labelled data. which means, 对于一些context，我们本应该是能得到对应的entity的，但是因为数据质量问题，这个entity在training data里面没有，那样的话，我把这些(context, question, answer) 也放到training data里面会对模型质量有影响么？