Apple M1/M2 Bug lists

[raymondlo84]

Keep tracking a set of bugs we face when we run OpenVINO 2022.3 (arm plugin) with OpenVINO Notebooks here. Since the arm_plugin is not officially supported and thus all fixes here are worked on as case by case only.

To learn more how to install OpenVINO on Apple Silicons, please read the blog below. https://raymondlo84.medium.com/how-to-install-openvino-notebooks-on-apple-m1-m2-silicons-a4c704747c12

[raymondlo84]

Known issues: 11/28/2022

1. Human pose estimations 402 is not working
2. NNCF dependencies in requirements.txt prevented the Notebooks to be installed properly and thus disabled.
3. Performance seems to be even lower than (Rosetta counter part) -- thus this will need to be confirmed.
4. No GPUs support is enabled at the moment.

[eaidova]

@raymondlo84 is it still actual? Probably N2 should be fixed now and nncf installation disabled for notebooks where it does not required, but if you still have any problem with that, let's discuss (possibly with nncf team involving)

what about problem 1? Can we help somehow?

3-4, I believe are not bugs, it is just specific of experimental arm platform support on openvino side (only cpu plugin enabled, backend libraries difference)

[raymondlo84]

I think the main stopper now is the tensorflow. with M1, we have to install tensorflow-macos prior running the pip install -r requirements.txt or we will get this error.

ERROR: Could not find a version that satisfies the requirement tensorflow<=2.9.3,>=2.5; extra == "tensorflow2" (from openvino-dev[onnx,tensorflow2]) (from versions: none) ERROR: No matching distribution found for tensorflow<=2.9.3,>=2.5; extra == "tensorflow2"

[raymondlo84]

pip install tensorflow-macos==2.9.2

So I have to run this or the dependencies checker will failed and stopped the installation. And also have to take out tensorflow2:

openvino-dev[onnx]==2022.3.0

[raymondlo84]

@adrianboguszewski -- you have to M1 now and please also feel free to repeat the work.

Also, we didn't specify if we shall use brew or just native python from xcode... So that's another tricky part as it will change our CI too.

[raymondlo84]

Also, I have to also preinstall onnx and pandas or it will try to build the wheels and fail again D:

Building wheels for collected packages: onnx, pandas
  Building wheel for onnx (setup.py) ... error
  error: subprocess-exited-with-error 

... So my workaround to try the first notebook was really skip the requirements.txt and install by hand one by one...

https://medium.com/@raymondlo84/how-to-run-openvino-2022-3-lts-on-apple-m1-40c38cf9a16f

[adrianboguszewski]

@raymondlo84 I tested the latest version of the repo on mac M1. I encountered the installation issues you experienced, but to deal with them, I updated the macOS wiki (https://github.com/openvinotoolkit/openvino_notebooks/wiki/macOS) and created a new PR (#766)

I see no issues with 402, but the overall performance is rather bad.

[raymondlo84]

Dynamic Shape will cause issues with ARM... We shall document that somewhere.

[raymondlo84]

406 won't run with M1...

[adrianboguszewski]

@raymondlo84 I tested all of them on my mac. 3/5 of them don't work. However, in most cases, we need to fix 1-2 bugs to enable the next 20.

The issues we have:

• get_shape error for dynamic shapes
• too old protobuf package
• hanging or extremely long quantization process
• not supported operators
• unknown crashes (to investigate)

1. get_shape issue:
   • 002-openvino-api
   • 004-hello-detection
   • 107-speech-recognition-quantization
   • 203-meter-reader
   • 204-named-entity-recognition
   • 208-optical-character-recognition
   • 211-speech-to-text
   • 214-vision-paddle-classification
   • 223-gpt2-text-prediction
   • 224-3D-segmentation-point-clouds
   • 227-whisper-subtitles-generation
   • 405-paddle-ocr-webcam
2. protobuf issue:
   • 104-model-tools
   • 207-vision-paddlegan-superresolution
   • 217-vision-deblur
   • 220-yolov5-accuracy-check-and-quantization
   • 222-vision-image-colorization
   • 226-yolov7-optimization
   • 230-yolo8-optimization
   • 406-3D-pose-estimation-webcam
3. long quantization issue:
   • 103-paddle-to-openvino
   • 105-language-quantize-bert
   • 110-ct-segmentation-quantize
   • 111-detection-quantization
   • 112-pytorch-post-training-quantization-nncf
   • 113-image-classification-quantization
   • 114-quantization-simplified-mode
   • 301-tensorflow-training-openvino
4. other issues:
   • 202-vision-superresolution (not supported operations)
   • 206-vision-paddlegan-anime (unknown crash)
   • 228-clip-zero-shot-image-classification (all devices failed)

[raymondlo84]

File these issues back on the contrib repos? And maybe also show this to Yury? He knows who can help there.

-ray

@raymondlo84 I tested all of them on my mac. 3/5 of them don't work. However, in most cases, we need to fix 1-2 bugs to enable the next 20.

The issues we have:

• get_shape error for dynamic shapes
• too old protobuf package
• hanging or extremely long quantization process
• not supported operators
• unknown crashes (to investigate)

1. get_shape issue:

• 002-openvino-api
• 004-hello-detection
• 107-speech-recognition-quantization
• 203-meter-reader
• 204-named-entity-recognition
• 208-optical-character-recognition
• 211-speech-to-text
• 214-vision-paddle-classification
• 223-gpt2-text-prediction
• 224-3D-segmentation-point-clouds
• 227-whisper-subtitles-generation
• 405-paddle-ocr-webcam

2. protobuf issue:

• 104-model-tools
• 207-vision-paddlegan-superresolution
• 217-vision-deblur
• 220-yolov5-accuracy-check-and-quantization
• 222-vision-image-colorization
• 226-yolov7-optimization
• 230-yolo8-optimization
• 406-3D-pose-estimation-webcam

3. long quantization issue:

• 103-paddle-to-openvino
• 105-language-quantize-bert
• 110-ct-segmentation-quantize
• 111-detection-quantization
• 112-pytorch-post-training-quantization-nncf
• 113-image-classification-quantization
• 114-quantization-simplified-mode
• 301-tensorflow-training-openvino

4. other issues:

• 202-vision-superresolution (not supported operations)
• 206-vision-paddlegan-anime (unknown crash)
• 228-clip-zero-shot-image-classification (all devices failed)

[raymondlo84]

This would be a good GSOC project. just to benchmark and cross validate ARM vs x86 with OpenVINO.

[eaidova]

protobuf issue

@adrianboguszewski could you please provide more details? protobuf<3.20 requirements comes from tensorflow and at this moment there is no package which supports protobuf 3.20.x.

as I can see, problem connected with onnx export, but I need to see some logs for better understanding can it be solved without protobuf upgrade or not

[raymondlo84]

Doesn't work with the stable diffusion too..

[adrianboguszewski]

@raymondlo84 that's interesting because if it's not on the list above, it means it worked for me

[raymondlo84]

@raymondlo84 that's interesting because if it's not on the list above, it means it worked for me

how...

[adrianboguszewski]

No idea how, but no problems. Reset your environment?