weitechen / anafora

Anafora is a web-based raw text annotation tool
241 stars 54 forks source link

Disagreement in an entity shows up as disagreements in entities that reference it #48

Open skatje-myers opened 7 years ago

skatje-myers commented 7 years ago

As you can see in ID 169 here (span "bends"), Anafora is claiming a disagreement in the "part_of" property, when they are actually referring to the same entity ("fault").

31240697-b92ed3ac-a9be-11e7-9063-8cadb805a31c

If we look at that entity, we can see that this is because the annotators disagreed on the properties of the "fault".

31240695-b80d6218-a9be-11e7-9b01-f4d5276e95e0

Anafora should mark the latter as a disagreement, but not the former. So perhaps comparing just span indices and entity types of the properties, not the properties of properties.

skatje-myers commented 6 years ago

Just hoping for an update on this sometime; it's a pretty serious issue.