search

hajicj / MUSCIMarker

Tool for annotating objects in musical scores.
http://muscimarker.readthedocs.io/en/latest/
Apache License 2.0
7 stars 2 forks source link

Implement inter-annotator agreement analysis #141

Open hajicj opened 7 years ago

hajicj commented 7 years ago

In order to evaluate the reliability of the dataset (and later of recognition systems), we need to measure the extent to which two annotations of the same image are similar.

hajicj commented 7 years ago

Alignment trouble:

Sometimes, two CropObjects overlap perfectly (e.g. a key signature if it only has one sharp). In this case, because of strict clsname checking, we should ensure that if there is a tie in pixel metrics, the CropObject with a matching clsname will be selected as the alignment pair.