search

wikilinks / neleval

Entity disambiguation evaluation and error analysis tool
Apache License 2.0
116 stars 23 forks source link

Deduplicate and remove mentions from excluded spans for TAC14 #13

Closed jnothman closed 9 years ago

jnothman commented 9 years ago

This mostly modifies prepare-tac and the scripts that call it.

I've not got tests covering these cases, which is probably bad, but I may merge soon in any case.

jnothman commented 9 years ago

@benhachey, you may want to check that you find the changes to scripts/run_* produce a condonable interface.

benhachey commented 9 years ago

Thanks for the ping. Interface change is fine by me. Please update the wiki page accordingly: https://github.com/wikilinks/neleval/wiki/Convenience

On Mon Feb 02 2015 at 1:45:37 PM jnothman notifications@github.com wrote:

@benhachey https://github.com/benhachey, you may want to check that you find the changes to scripts/run_* produce a condonable interface.

— Reply to this email directly or view it on GitHub https://github.com/wikilinks/neleval/pull/13#issuecomment-72399976.

jnothman commented 9 years ago

As per latest NIST discussions, it seems we should make this optional in the script... More fun.

jnothman commented 9 years ago

Okay. In the launcher scripts I've made providing excluded spans optional by having any additional arguments beyond the basic ones be passed to prepare-tac. This may be a bit user-unfriendly (command-line arg parsing is now less strict), but seems a decent mid-way solution.

jnothman commented 9 years ago

This also no longer invalidates former command-line usage of the run_tac* scripts, so I'm happier to merge it ASAP.

jnothman commented 9 years ago

To avoid putting this same feature in the run_tac*_filtered scripts, those now only take a single arg, the output dir of a whole evaluation.

jnothman commented 9 years ago

Ready to merge.