BreezeWhite / oemer

End-to-end Optical Music Recognition (OMR) system. Transcribe phone-taken music sheet image into MusicXML, which can be edited and converted to MIDI.
https://breezewhite.github.io/oemer/
MIT License
386 stars 46 forks source link

No tenor and alto clef ? #27

Closed LmYjQ closed 1 year ago

LmYjQ commented 1 year ago

I want to recognize some viola scores with tenor clef, but there are only treble clef in the result. I guess there are no tenor or alto clef in the training data?

BreezeWhite commented 1 year ago

Hi @LmYjQ , yes you're correct, tenor clef is not included in the pipeline. The whole system is designed to recognize piano sheet, therefore other kinds of music sheet of instrument may not work.

LmYjQ commented 1 year ago

@BreezeWhite OK. If I want to add the function to recognize tenor clef, I think I only need to change the SVM model? In detail, 1.Collect some scores with tenor clef. 2.Extract the symbol with the existing Unet model. 3.Retrain and replace the SVM model.

It might be very hard to distinguish the tenor and alto clef, cause they have the same shape, only different position on the staff.Do you have some idea about this?

BreezeWhite commented 1 year ago

Not only the SVM model, but also the two UNet models need to be fine-tuned, since the "shape" of these clef types is also new to the models.

It's actually not that hard to distinguish the two clef types. It's the same case like determining the pitch of a notehead, you can decide the clef type by comparing the position relative to the stafflines. After getting the correct type, the following inference process also need to be adjusted to consider two more clef cases, since the clef symbol will apparently affect the pitch of each note.

LmYjQ commented 1 year ago

Thank you. I'll try.