dice-group / gerbil

GERBIL - General Entity annotatoR Benchmark
GNU Affero General Public License v3.0
224 stars 58 forks source link

[QA] C2KB sub experiment takes all Meaning instances into account, including properties #455

Open MichaelRoeder opened 3 months ago

MichaelRoeder commented 3 months ago

Problem

The GERBIL QA paper describes the C2KB task to take all resources into account. The running example clarifies, that it only takes entities into account. However, in the code, properties are taken into account as well.

The problem stems from the class hierarchy. C2KB looks for instances of Meaning. However, a question has Annotation instances for entities and Property instances for properties. The latter are a subclass of the Annotation.

Solution