BreezeWhite / oemer

End-to-end Optical Music Recognition (OMR) system. Transcribe phone-taken music sheet image into MusicXML, which can be edited and converted to MIDI.
https://breezewhite.github.io/oemer/
MIT License
399 stars 46 forks source link

Unable to compute the prediction using a neural network model. #62

Open pehrlich2 opened 5 months ago

pehrlich2 commented 5 months ago

Describe the bug

Hi - fresh install crashes here. (Had to force numpy=1.26.4 due to what appears to be a bug in the newly released 2.0.0). Why trying to OCR a png:

Running OSX 14.0 on an M2 Macbook Air.

Input Image

songtosing

Full Traceback

 % oemer -d songtosing.png
2024-06-23 13:45:42 Extracting staffline and symbols
/Users/peterehrlich/projects/omr/lib/python3.12/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:69: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'CoreMLExecutionProvider, AzureExecutionProvider, CPUExecutionProvider'
  warnings.warn(
2024-06-23 13:45:42.915652 [W:onnxruntime:, coreml_execution_provider.cc:104 GetCapability] CoreMLExecutionProvider::GetCapability, number of partitions supported by CoreML: 86 number of nodes in the graph: 1577 number of nodes supported by CoreML: 102
1906 1928
Context leak detected, msgtracer returned -1
Context leak detected, msgtracer returned -1
2024-06-23 13:46:23.331865 [E:onnxruntime:, sequential_executor.cc:516 ExecuteKernel] Non-zero status code returned while running CoreML_18203898718667227551_0 node. Name:'CoreMLExecutionProvider_CoreML_18203898718667227551_0_0' Status Message: Error executing model: Unable to compute the prediction using a neural network model. It can be an invalid input data or broken/unsupported model (error code: -1).
Traceback (most recent call last):
  File "/Users/peterehrlich/projects/omr/bin/oemer", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/peterehrlich/projects/omr/lib/python3.12/site-packages/oemer/ete.py", line 276, in main
    mxl_path = extract(args)
               ^^^^^^^^^^^^^
  File "/Users/peterehrlich/projects/omr/lib/python3.12/site-packages/oemer/ete.py", line 127, in extract
    staff, symbols, stems_rests, notehead, clefs_keys = generate_pred(str(img_path), use_tf=args.use_tf)
                                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/peterehrlich/projects/omr/lib/python3.12/site-packages/oemer/ete.py", line 47, in generate_pred
    staff_symbols_map, _ = inference(
                           ^^^^^^^^^^
  File "/Users/peterehrlich/projects/omr/lib/python3.12/site-packages/oemer/inference.py", line 69, in inference
    out = model.predict(batch) if use_tf else sess.run(output_names, {'input': batch})[0]
                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/peterehrlich/projects/omr/lib/python3.12/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 220, in run
    return self._sess.run(output_names, input_feed, run_options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Command You Execute

oemer -d songtosing.png
liebharc commented 4 months ago

Hi, I wasn't able to reproduce this issue on a Linux machine. But possibly there is also a breaking change in onnxruntime. I will paste the output of my pip list below, hoping that this helps you to track down the issue. An alternative would be to install tensorflow with pip install .[tf] and use the --use-tf flag for running oemer. Here you also have to pin the tensorflow version to 2.15.1 as newer versions fail to load the oemer models.

For all these version issues, it might be good if there would be some kind of lock-file such as poetry has it so that users can install exactly the versions which oemer requires.

`pip list` ``` Package Version ----------------------- --------------- argcomplete 1.8.1 Babel 2.8.0 blinker 1.4 certifi 2022.12.7 cffi 1.15.1 click 8.0.3 colorama 0.4.4 coloredlogs 15.0.1 command-not-found 0.3 contourpy 1.2.1 crypto 1.4.1 cryptography 39.0.1 cycler 0.12.1 dbus-python 1.2.18 distro 1.7.0 distro-info 1.1build1 ecdsa 0.18.0 envs 1.4 exceptiongroup 1.2.1 flatbuffers 24.3.25 fonttools 4.53.1 gamera 4.1.0 httplib2 0.20.2 humanfriendly 10.0 idna 3.4 importlib-metadata 4.6.4 iniconfig 2.0.0 jeepney 0.7.1 Jinja2 3.0.3 jmespath 1.0.1 joblib 1.4.2 keyring 23.5.0 kiwisolver 1.4.5 lark-parser 0.7.7 launchpadlib 1.10.16 lazr.restfulclient 0.14.4 lazr.uri 1.0.6 livereload 2.6.3 Markdown 3.3.6 MarkupSafe 2.0.1 matplotlib 3.9.1 mkdocs 1.1.2 mock 4.0.3 more-itertools 8.10.0 mpmath 1.3.0 Naked 0.1.32 netifaces 0.11.0 numpy 1.26.4 oauthlib 3.2.0 oemer 0.1.7 onnxruntime-gpu 1.18.1 opencv-python-headless 4.10.0.84 packaging 21.3 pillow 10.4.0 pip 22.0.2 pipx 1.0.0 pluggy 1.5.0 protobuf 5.27.2 psutil 5.9.0 pyasn1 0.4.8 pycognito 2022.8.0 pycparser 2.21 Pygments 2.11.2 PyGObject 3.42.1 pyinotify 0.9.6 PyJWT 2.3.0 pyparsing 2.4.7 pytest 8.2.2 python-apt 2.3.0+ubuntu2.1 python-dateutil 2.8.2 python-jose 3.3.0 pytz 2022.1 PyYAML 6.0 rsa 4.9 scikit-learn 1.5.1 scipy 1.14.0 screen-resolution-extra 0.0.0 SecretStorage 3.3.1 setuptools 59.6.0 shellescape 3.8.1 six 1.16.0 sympy 1.13.1 systemd-python 234 threadpoolctl 3.5.0 tomli 2.0.1 tornado 6.1 types-Pillow 10.2.0.20240520 types-protobuf 5.27.0.20240626 types-requests 2.32.0.20240712 types-tensorflow 2.16.0.20240627 typing_extensions 4.12.2 ubuntu-advantage-tools 27.12 ufw 0.36.1 unattended-upgrades 0.1 urllib3 2.2.2 userpath 1.8.0 version-parser 1.0.1 wadllib 1.3.6 wheel 0.37.1 xkit 0.0.0 zipp 1.0.0 ```