Evaluation Measures - Githubissues

cosmir / openmic-annotator

Annotation framework for annotating data for OpenMIC

MIT License

56 stars 1 forks source link

Evaluation Measures #28

Open julian-urbano opened 7 years ago

julian-urbano commented 7 years ago

I made a draft gdoc with the description of the task we came to in #19, as well as the definition of evaluation measures for tasks 1 (classification) and 2 (retrieval). All this information will be available from the challenge's website http://cosmir.github.io/open-mic/

Each task has measures that ignore the instrument taxonomy and measures that take it into account.

Please take a look and leave comments before Nov 2nd at https://docs.google.com/document/d/1RF9aUCFh6_-YnoZfM6XhANZbGxQcP84B32saJCKyD3E/edit?usp=sharing

julian-urbano commented 7 years ago

Details to finalize:

[ ] Description of data and annotations.
[ ] Description of taxonomy.
[ ] Size of system output for task 2.