I noticed that in the datasets of relation classification, there is no entity pair of 'N/A' label (no relation between two entities). In the other words, entities without any relations are not considered in training and inference. For example, as I check in the dataset of SCIIE, I find only the six relations are considered: EVALUATE-FOR, PART-OF, USED-FOR, FEATURE-OF, CONJUNCTION, COMPARE, HYPONYM-OF. So basically in the evaluation, SciBERT takes a pair of entities that is assumed to to have an existing relation and classifies.
But obviously SCIIE have a lot of entity pairs that do not have any of the six relations, and they are not considered at inference time in this case. This is very different from other works that extract relations with gold entities, which do not assume a pair of input entities would have an existing relation beforehand. Specifically, they make inference for all possible pairs of entities, where the relation is classified over ( defined relations types + N/A relation) .
Could you clarify why N/A relation is not included in the training and the evaluation of SciBERT? Or am I missing something?
I noticed that in the datasets of relation classification, there is no entity pair of 'N/A' label (no relation between two entities). In the other words, entities without any relations are not considered in training and inference. For example, as I check in the dataset of SCIIE, I find only the six relations are considered: EVALUATE-FOR, PART-OF, USED-FOR, FEATURE-OF, CONJUNCTION, COMPARE, HYPONYM-OF. So basically in the evaluation, SciBERT takes a pair of entities that is assumed to to have an existing relation and classifies.
But obviously SCIIE have a lot of entity pairs that do not have any of the six relations, and they are not considered at inference time in this case. This is very different from other works that extract relations with gold entities, which do not assume a pair of input entities would have an existing relation beforehand. Specifically, they make inference for all possible pairs of entities, where the relation is classified over ( defined relations types + N/A relation) .
Could you clarify why N/A relation is not included in the training and the evaluation of SciBERT? Or am I missing something?