Given the number of variant tasks, we need to document the task setting for each system, in a structured form. Ideally, we would like to know:
does the system take gold mentions as input?
if so, do they include gold nils?
does the system output intend to reflect exactly CoNLL 2003 entities (PER, LOC, ORG, MISC) or does it perform additional wikification? (perhaps: does it annotate only a subset of PER, LOC, ORG?)
what objective metric / corpus was used to develop/optimise the system?
We should additionally encourage README files including bibtex references.
Finally, I'm not sure that .analysis files should be stored in the repo.
Given the number of variant tasks, we need to document the task setting for each system, in a structured form. Ideally, we would like to know:
We should additionally encourage README files including bibtex references.
Finally, I'm not sure that
.analysisfiles should be stored in the repo.