Differences in performance (DBpedia spotlight)

dice-group / gerbil

GERBIL - General Entity annotatoR Benchmark

GNU Affero General Public License v3.0

222 stars 58 forks source link

Differences in performance (DBpedia spotlight) #244

Closed alexanderpanchenko closed 6 years ago

alexanderpanchenko commented 6 years ago

I am curious about difference in results between the DBpedia annotator and the NIF based web service (available in a GERBIL branch) based on the DBPedia annotator as presented below:

I registered a new NIF based web service, which is available in the branch as explained here: https://github.com/dice-group/gerbil/wiki/How-to-create-a-NIF-based-web-service.

The difference is very big but I would expect the results to be the same (as this is a simple wrapper). Is there any issue with my setup or I misunderstand something?

alexanderpanchenko commented 6 years ago

When one selects A2KB experiment numbers are the same (so I will close this ticket), but still interesting to figure out why for D2KB they are not (I guess because NER annotation is ignored)P:

MichaelRoeder commented 6 years ago

The two services are not exactly the same. The NIF-based web service example (https://github.com/dice-group/gerbil/blob/SpotWrapNifWS4Test/src/main/java/org/aksw/gerbil/ws4test/SpotlightClient.java#L48) uses a different service than the service that we are using for DBpedia Spotlight (https://github.com/dice-group/gerbil/blob/master/src/main/properties/annotators.properties#L50).

alexanderpanchenko commented 6 years ago

ok, i see. thank you