Open mertalev opened 3 months ago
+@sfatimar , @preetha-intel
Having the same error using this model: https://huggingface.co/SmilingWolf/wd-swinv2-tagger-v3 Other variants of this model (convnext and vit) runs fine but swinv2 fails with the following logs:
Using onnxruntime-openvino==1.17.1
on Arch Linux 6.8.2 with this script:
https://github.com/kohya-ss/sd-scripts/blob/dev/finetune/tag_images_by_wd14_tagger.py
Command:
ORT_OPENVINO_ENABLE_CI_LOG=1 ORT_OPENVINO_ENABLE_DEBUG=1 OPENVINO_LOG_LEVEL=5 python finetune/tag_images_by_wd14_tagger.py --model_dir "~/models/wd14_tagger_model" --repo_id "SmilingWolf/wd-swinv2-tagger-v3" --recursive --remove_underscore --append_tags --onnx --caption_separator ", " --batch_size 1 --caption_extension ".txt" point_to_a_folder_with_images/
On CPU with OpenVINO {'device_type': 'CPU_FP32'}
):
On an Intel ARC A770 ({'device_type': 'GPU.0_FP32'}
or {'device_type': 'GPU_FP32'}
):
On an AMD RX 7900 XTX {'device_type': 'GPU.1_FP32'}
):
(Different error and more useful logs)
Same script runs fine on CPUExecutionProvider
and ROCmExecutionProvider
(from https://pypi.lsh.sh/60/onnxruntime-training/).
Yes there is a regression Binary DLL uploaded in github.com/intel/onnxruntime for 1.17.1 onnxruntime-openvino is compatible only with OpenVINO 2023.3.0. There was a change in Exception API on OpenVINO which was not handled properly so backward compatibility was broken. But it is possible to build 1.17.1 code with OpenVINO 2023.1.0 and execute ...
I tried buıilding from the main branch (commit id: a2998e5d425d48f83f817f6503c6bd196e2cb3ae) and it runs fine now.
Had to use OpenVINO 2023.3 since building with 2024.0 segfaulted on import.
2023.3 still fails when running tests but it runs fine for my use case so i added --skip_tests
to build.sh.
Ran into the same issue as https://github.com/intel/neural-speed/issues/188 and had to add --compile_no_warning_as_error
to build.sh.
Build command:
./build.sh --config RelWithDebInfo --use_openvino GPU_FP32 --parallel --build_shared_lib --build_wheel --compile_no_warning_as_error --skip_tests
Yes there is a regression Binary DLL uploaded in github.com/intel/onnxruntime for 1.17.1 onnxruntime-openvino is compatible only with OpenVINO 2023.3.0.
There was a change in Exception API on OpenVINO which was not handled properly so backward compatibility was broken. But it is possible to build 1.17.1 code with OpenVINO 2023.1.0 and execute ...
To clarify, the issue is occurring with 2023.3.0. The pattern I'm seeing is that it works on CPUs with Iris Xe graphics, but not on CPUs with UHD graphics.
Is it possible to DOWNGRADE to 1.15.0 onnxruntime-openvino with OpenVINO 2023.1.0? If yes, how with docker compose (sorry but I"m not an expert) Thanks
I have to confirm, that this is an existing issue that breaks OpenVINO using Intel Arc (770) under Windows.
@shummo A downgrade to onnxruntime==1.15.0
and openvino=2023.1.0
solved it
Hi @mertalev, I have tested your model this and it's inferencing successfully on both Windows 11 and Ubuntu 22.04 for CPU and GPU. I would recommend you and community towards using latest OpenVINO Toolkit v2024.1 and OpenVINO EP v1.18.0 which will be available soon in the upcoming ONNXRuntime release. You can also build and find OpenVINO EP from source for the same.
Thanks for the testing and update! We'll upgrade to 1.18.0 and 2024.1.0 once the former is available.
When you mention that it works on GPU, can you clarify if you tested with an iGPU like UHD Graphics, or a dGPU like Arc (and I believe Iris Xe is also counted as a dGPU). iGPUs struggle, but I haven't seen anyone with a dGPU have an issue with this model.
Thanks for the testing and update! We'll upgrade to 1.18.0 and 2024.1.0 once the former is available.
When you mention that it works on GPU, can you clarify if you tested with an iGPU like UHD Graphics, or a dGPU like Arc (and I believe Iris Xe is also counted as a dGPU). iGPUs struggle, but I haven't seen anyone with a dGPU have an issue with this model.
It's tested on a Meteor Lake architecture processor CPU (Intel Core Ultra 7 1003H) comprising of iGPU (Intel Arc Graphics). I'd recommend you to try your application on multiple platforms. As suggested above, you can also use the main branch of this repository to build OpenVINO EP to get the latest wheels for your work environment and reproduce the same.
Thanks for the update!
It's tested on a Meteor Lake architecture processor CPU (Intel Core Ultra 7 1003H) comprising of iGPU (Intel Arc Graphics).
Hi, could you please add Intel UHD graphics to your list of test cases/testing setup because it is currently broken on that (but not XE graphics, so if it works on your testing setup, this may still be broken)
@ankitm3k No, onnxruntime-openvino does not work with latest OpenVino 2024.1 ... just release 1.18.0 finally
Thanks for the testing and update! We'll upgrade to 1.18.0 and 2024.1.0 once the former is available.
When you mention that it works on GPU, can you clarify if you tested with an iGPU like UHD Graphics, or a dGPU like Arc (and I believe Iris Xe is also counted as a dGPU). iGPUs struggle, but I haven't seen anyone with a dGPU have an issue with this model.
Hi @mertalev @Snuupy
I have tested your model with our C++ onnxruntime_perf_test app built from source and it runs inference successfully with below machine configurations - Machine 1 - OS: Windows 11 CPU: Raptor Lake arch iGPU: Intel UHD Graphics 770 dGPU: Intel Arc A380 Graphics
Machine 2 - OS: Windows 11 CPU: i7-1270P iGPU: Intel Iris Xe Graphics
I'd recommend you to use either intel's repo master or rel-1.18.0 (https://github.com/intel/onnxruntime.git) or directly from Microsoft's master branch (https://github.com/microsoft/onnxruntime.git) or rel-1.18.0 (https://github.com/microsoft/onnxruntime/commits/rel-1.18.0/) for building the wheels using below command -
Discover the wheel in below example path and install using pip - pip install ${CWD}\onnxruntime\build\Windows\Debug\Debug\dist\onnxruntime_openvino-1.19.0-cp310-cp310-win_amd64.whl
@ankitm3k
I have tested your model with our C++ onnxruntime_perf_test app built from source and it runs inference successfully with below machine configurations - Machine 1 - OS: Windows 11 CPU: Raptor Lake arch iGPU: Intel UHD Graphics 770 dGPU: Intel Arc A380 Graphics
Doesn't onnxruntime default to the dgpu if one is provided? So it will run on the A380 (working) instead of the UHD 770 igpu (not working)
I can try substituting 1.18 to see if that fixes anything regardless.
@ankitm3k
I have tested your model with our C++ onnxruntime_perf_test app built from source and it runs inference successfully with below machine configurations - Machine 1 - OS: Windows 11 CPU: Raptor Lake arch iGPU: Intel UHD Graphics 770 dGPU: Intel Arc A380 Graphics
Doesn't onnxruntime default to the dgpu if one is provided? So it will run on the A380 (working) instead of the UHD 770 igpu (not working)
I can try substituting 1.18 to see if that fixes anything regardless.
The onnxruntime default device_type for GPU is iGPU (GPU.0) and if you want to explicitly use dGPU (GPU.1) then set your device _type as GPU.1 during your inference provider options.
Please build your onnxruntime-openvino wheels from the main branch and install it in your python virtual env so that you can get the latest release changes. This should solve your issues.
Describe the issue
When using OpenVINO, the session can be created, but calling
run
leads to the error:RuntimeException: [ONNXRuntimeError] RUNTIME_EXCEPTION: Encountered unknown exception
. Based on reports in this issue, there seems to be a pattern with the N100 CPU in particular.This seems to be a regression as this error only appears after upgrading to 1.17.1 of onnxruntime-openvino with OpenVINO 2023.3.0. This model worked when using 1.15.0 and OpenVINO 2023.1.0.
After enabling the following environmental variables:
There are a few additional logs, but none that seem pertinent:
To reproduce
With onnxruntime-openvino 1.17.1 and OpenVINO 2023.3.0, create a session including the following providers:
And the following provider options:
Then attempt to run inference with this model. It may or may not work depending on the CPU.
You may use this image to have the exact software environment producing the issue:
ghcr.io/immich-app/immich-machine-learning@sha256:01799596c7f40495887d4027df1c0f4c144c7cd6ab34937ef2cc14d246470095
Urgency
No response
Platform
Linux
OS Version
Ubuntu 22.04
ONNX Runtime Installation
Released Package
ONNX Runtime Version or Commit ID
1.17.1
ONNX Runtime API
Python
Architecture
X64
Execution Provider
OpenVINO
Execution Provider Library Version
2023.3.0