search

BreezeWhite / oemer

End-to-end Optical Music Recognition (OMR) system. Transcribe phone-taken music sheet image into MusicXML, which can be edited and converted to MIDI.
https://breezewhite.github.io/oemer/
MIT License
394 stars 46 forks source link

AssertionError at dewarp.py, line 198 #43

Closed EwingKang closed 1 year ago

EwingKang commented 1 year ago

Describe the bug AssersionError with this specific image

Input Image Edit :removed copyrighted image, contact me privately (Note: I use the tiff version directly from the scanner, which explains the discrepancy between command and logs, I'm able to recreate this error with this uploaded PNG version)

Full Traceback

user@laptop:~/omr_ws/oemer$ PYTHONPATH=$(pwd) LOG_LEVEL=debug python3 oemer/ete.py ../sample_img/IMG_0005.tif --save-cache
2023-10-09 14:38:09 [INFO] Dewarping  [at ete.py:149]
2023-10-09 14:38:09 [DEBUG] Building grids  [at dewarp.py:276]
2023-10-09 14:38:12 [DEBUG] Labeling areas  [at dewarp.py:279]
2023-10-09 14:38:13 [DEBUG] Connecting lines  [at dewarp.py:282]
Traceback (most recent call last):
  File "/home/ewing/omr_ws/oemer/oemer/ete.py", line 286, in <module>
    main()
  File "/home/ewing/omr_ws/oemer/oemer/ete.py", line 280, in main
    mxl_path = extract(args)
  File "/home/ewing/omr_ws/oemer/oemer/ete.py", line 150, in extract
    coords_x, coords_y = estimate_coords(staff)
  File "/home/ewing/omr_ws/oemer/oemer/dewarp.py", line 283, in estimate_coords
    new_gg_map = connect_nearby_grid_group(gg_map, grid_groups, grid_map, grids)
  File "/home/ewing/omr_ws/oemer/oemer/dewarp.py", line 198, in connect_nearby_grid_group
    assert grid_id in grid_groups[label].gids, f"{grid_id}, {label}"
AssertionError: -1, -1

Command You Execute PYTHONPATH=$(pwd) LOG_LEVEL=debug python3 oemer/ete.py ../sample_img/IMG_0005.tif --save-cache

EwingKang commented 1 year ago

I've been doing some bug tracing. Some related values are printed when pdb caught the assertion:

(Pdb) overlapped_size
[(-1, 22), (57, 22), (280, 22)]
(Pdb) unique
array([ -1,  57, 280], dtype=int32)
(Pdb) region
array([[ 57,  57,  57,  57,  57,  57,  57,  57,  57,  57,  57],
       [ 57,  57,  57,  57,  57,  57,  57,  57,  57,  57,  57],
       [ -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1],
       [ -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1],
       [280, 280, 280, 280, 280, 280, 280, 280, 280, 280, 280],
       [280, 280, 280, 280, 280, 280, 280, 280, 280, 280, 280]],
      dtype=int32)

It seems like ID: -1 is taken into account when there are multiple nearby groups. I'll try adding some patch to work around the issue.

BreezeWhite commented 1 year ago

The PR has been merged.