openvinotoolkit / openvino

OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference
https://docs.openvino.ai
Apache License 2.0
6.83k stars 2.18k forks source link

openVINO2024.3.0 gave RuntimeError:[GPU] ProgramBuilder build failed! at notebook:3D-segmentation-point-clouds, when using Integrated GPU #26324

Open YOODS-Xu opened 1 month ago

YOODS-Xu commented 1 month ago

Describe the bug openVINO2024.3.0 gave RuntimeError:[GPU] ProgramBuilder build failed! at notebook:3D-segmentation-point-clouds, when using Integrated GPU both 11 Gen corei7 and 13 Gen corei9. openVINO2023.3.0 worked well but integrated GPU were slower than CPUs.

Expected behavior Did any other's openVINO2024.3.0 work well at notebook:3D-segmentation-point-clouds, when using Integrated GPU? What is the matter with me, or is it dispossible? How could my Integrated GPU be faster than CPU using segmetation models such as 3D-segmentation-point-clouds and Mask rcnn?

Screenshots screenshot

Installation instructions (Please mark the checkbox) [yes] I followed the installation guide at https://github.com/openvinotoolkit/openvino_notebooks#-installation-guide to install the notebooks.

Environment information Only next is not ok, but I donot know how to fix it, and if it mattered. Jupyter kernel installed for openvino_env:

print(f"kernel spec={KernelSpecManager().get_all_specs()}") gave as below. kernel spec={'python3': {'resource_dir': '/home2/u22-ros/venvs/openvino_env/share/jupyter/kernels/python3', 'spec': {'argv': ['python', '-m', 'ipykernel_launcher', '-f', '{connection_file}'], 'env': {}, 'display_name': 'Python 3 (ipykernel)', 'language': 'python', 'interrupt_mode': 'signal', 'metadata': {'debugger': True}}}}

After modified kernel = KernelSpecManager().get_kernel_spec("openvino_env") to kernel = KernelSpecManager().get_kernel_spec("python3") then below are not ok. Jupyter kernel Python and OpenVINO environment Python match: NOT OK

Additional context Add any other context about the problem here.

brmarkus commented 1 month ago

I just tried to reproduce the Juypter notebook "notebooks/3D-segmentation-point-clouds/3D-segmentation-point-clouds.ipynb". On a MS-Win 11 Laptop (Intel Core Ultra 7 155H). With a clean Python venv, using current HEAD-revision of this notebooks repo. I can confirm the inference is working when using the CPU. But it fails when using the GPU. (NPU not working due to use of dynamic shapes)

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[14], line 2
      1 # Inference
----> 2 compiled_model = core.compile_model(model=model, device_name=device.value)
      3 output_layer = compiled_model.output(0)
      4 result = compiled_model([point])[output_layer]

File [C:\localdisk\OpenVINO-MSWin\openvino_env\Lib\site-packages\openvino\runtime\ie_api.py:543](file:///C:/localdisk/OpenVINO-MSWin/openvino_env/Lib/site-packages/openvino/runtime/ie_api.py#line=542), in Core.compile_model(self, model, device_name, config, weights)
    538     if device_name is None:
    539         return CompiledModel(
    540             super().compile_model(model, {} if config is None else config),
    541         )
    542     return CompiledModel(
--> 543         super().compile_model(model, device_name, {} if config is None else config),
    544     )
    545 else:
    546     if device_name is None:

RuntimeError: Exception from src\inference\src\cpp\core.cpp:107:
Exception from src\inference\src\dev\plugin.cpp:53:
Check 'false' failed at src\plugins\intel_gpu\src\plugin\program_builder.cpp:185:
[GPU] ProgramBuilder build failed!
Program build failed(0_part_11):
21:5264:5: error: token is not a valid binary operator in a preprocessor subexpression
#if INPUT_LEFTOVERS
    ^~~~~~~~~~~~~~~
21:5133:38: note: expanded from macro 'INPUT_LEFTOVERS'
....
21:7301:5: error: token is not a valid binary operator in a preprocessor subexpression
#if INPUT_LEFTOVERS
    ^~~~~~~~~~~~~~~
21:7170:38: note: expanded from macro 'INPUT_LEFTOVERS'
....
21:9338:5: error: token is not a valid binary operator in a preprocessor subexpression
#if INPUT_LEFTOVERS
    ^~~~~~~~~~~~~~~
21:9207:38: note: expanded from macro 'INPUT_LEFTOVERS'
....

According to a call to %pip freeze I got OpenVINO installed in version openvino==2024.3.0... your reported problem seems to exist with older versions already.

YOODS-Xu commented 3 weeks ago

@brmarkus Thank you very much for your test. I am so sorry, I took a miss about the versions. It should be 24.3.0(fast but error) and 23.3.0(correct but slow). Hope Intel could give us an answer.