search

wikilinks / conll03_nel_eval

Python evaluation scripts for AIDA-formatted CoNLL data
Apache License 2.0
20 stars 4 forks source link

Implement TAC NIL clustering measures #38

Closed benhachey closed 10 years ago

benhachey commented 10 years ago

CEAF, B-cubed, pairwise PRF

jnothman commented 10 years ago

These plus the MUC score as defined in Vilain et al (1995) are implemented at https://github.com/jnothman/conll03_nel_eval/compare/ceaf albeit needing more testing for edge cases. For instance Cai and Strube's (2010) adjustments have not been included, and they have been belatedly in the Pradhan et al. CoNLL scorer.

While implemented as metrics over sets, these have not yet been integrated into the evaluator.

On 14 May 2014 10:01, Ben Hachey notifications@github.com wrote:

CEAF, B-cubed, pairwise PRF

— Reply to this email directly or view it on GitHubhttps://github.com/wikilinks/conll03_nel_eval/issues/38 .

benhachey commented 10 years ago

Integration: #39 Testing: #43