search

BeritJanssen / PatternsForPrediction

Scripts for evaluation and conversion of files in Patterns for Prediction
2 stars 3 forks source link

Handling polyphonic submissions for the monophonic task #5

Closed JamesOwers closed 5 years ago

JamesOwers commented 5 years ago

Lines 57 and 58 in ./evaluate_prediction are unused. https://github.com/BeritJanssen/PatternsForPrediction/blob/master/evaluate_prediction.py#L57

Further down the file, the onsets are extracted again and, this time, set is not used to deduplicate them. In the case of the monophonic task, if this file was used as is for evaluation, the results may be incorrect if generated submissions were not monophonic (because the number of elements in the set Q will be larger => precision will be lower).

It is not stated that there should be a deduplication in the score as described here. In the monophonic case, how would a polyphonic submission be handled? This would matter for the pitch scores; should both pitches be included if simultaneous notes were submitted?

BeritJanssen commented 5 years ago

I tested the code for monophonic and polyphonic submissions. If there is a polyphonic continuation to a monophonic prime, I would consider it fair that this gets lower precision.