hipe-eval / HIPE-scorer

A python module for evaluating NERC and NEL system performances as defined in the HIPE shared tasks (formerly CLEF-HIPE-2020-scorer).
https://hipe-eval.github.io
MIT License
13 stars 4 forks source link

evaluation NEL with multiple links #7

Closed aflueckiger closed 4 years ago

aflueckiger commented 4 years ago

@maud @e-maud @simon-clematide

With https://github.com/impresso/CLEF-HIPE-2020-scorer/commit/c457b7f439f04862f7dccc18aba9aad587df4e12, I added the two options to evaluate on a multitude of links:

There are some limitations:

The limations are not due to technical problems but rather to keep it as simple as possible. Our scorer grew in complexity already considerably due to our advanced evaluation scenarios.

aflueckiger commented 4 years ago

Remove unionizing feature as of https://github.com/impresso/CLEF-HIPE-2020-scorer/commit/f23c50cdbf76875bbbea2f4eddeda9182ee70cfa to keep the script as clean and tidy as possible. The same behaviour can be easily achieved with a pre-processing step merging both columns in a ranked list.