[Ubuntu 22.04] RuntimeError: Failed to create plugin [path]/libopenvino_intel_gpu_plugin.so for device GPU

[andyyeh75]

Hi team,

Encountered an error for choosing GPU as the device for inference while clicked the load models.

RuntimeError: Failed to create plugin /home/u/openvino-ai-plugins-gimp/gimpenv3/lib/python3.10/site-packages/openvino/libs/libopenvino_intel_gpu_plugin.so for device GPU

I do have perform the openvino installation guide. but error still occurs. apt-get install -y ocl-icd-libopencl1 intel-opencl-icd intel-level-zero-gpu level-zero

full log while pressing "load models" with GPU device as attached:

`flag_openpose True /home/u/openvino-ai-plugins-gimp/weights/../gimp_openvino_run_sd.json not found, loading defaults

(stable-diffusion-ov.py:183): Gtk-CRITICAL **: 09:49:38.491: gtk_box_pack: assertion '_gtk_widget_get_parent (child) == NULL' failed

(stable-diffusion-ov.py:183): Gtk-CRITICAL **: 09:49:38.491: gtk_box_pack: assertion '_gtk_widget_get_parent (child) == NULL' failed

(stable-diffusion-ov.py:183): Gtk-CRITICAL : 09:49:38.493: gtk_box_pack: assertion '_gtk_widget_get_parent (child) == NULL' failed No stable-diffusion model server found to kill Exception in thread Thread-1 (async_load_models): Traceback (most recent call last): File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner self.run() File "/usr/lib/python3.10/threading.py", line 953, in run self._target(*self._args, *self._kwargs) File "/home/u/openvino-ai-plugins-gimp/gimpenv3/lib/python3.10/site-packages/gimpopenvino/plugins/stable-diffusion-ov/stable-diffusion-ov.py", line 353, in async_load_models s.bind((HOST, 65433)) OSError: [Errno 98] Address already in use /home/u/openvino-ai-plugins-gimp/gimpenv3/lib/python3.10/site-packages/gimpopenvino/tools/openvino_common/models_ov/stable_diffusion_engine_NEW.py:22: FutureWarning: Importing DiffusionPipeline or ImagePipelineOutput from diffusers.pipeline_utils is deprecated. Please import from diffusers.pipelines.pipeline_utils instead. from diffusers.pipeline_utils import DiffusionPipeline /home/u/openvino-ai-plugins-gimp/gimpenv3/lib/python3.10/site-packages/controlnet_aux/open_pose/body.py:7: DeprecationWarning: Please use gaussian_filter from the scipy.ndimage namespace, the scipy.ndimage.filters namespace is deprecated. from scipy.ndimage.filters import gaussian_filter /home/u/openvino-ai-plugins-gimp/gimpenv3/lib/python3.10/site-packages/controlnet_aux/open_pose/hand.py:4: DeprecationWarning: Please use gaussian_filter from the scipy.ndimage namespace, the scipy.ndimage.filters namespace is deprecated. from scipy.ndimage.filters import gaussian_filter /home/u/openvino-ai-plugins-gimp/gimpenv3/lib/python3.10/site-packages/controlnet_aux/mediapipe_face/mediapipe_face_common.py:7: UserWarning: The module 'mediapipe' is not installed. The package will have limited functionality. Please install it using the command: pip install 'mediapipe' warnings.warn( [ INFO ] Model Name: SD_1.4 [ INFO ] Initializing Inference Engine... [ INFO ] Model Path: /home/u/openvino-ai-plugins-gimp/weights/stable-diffusion-ov/stable-diffusion-1.4 [ INFO ] device_name: ['GPU', 'GPU', 'GPU'] Looking for GIMP process Done looking for GIMP process gimp-2.99 process found: psutil.Process(pid=2, name='gimp-2.99', status='sleeping', started='09:44:17') Text Device: GPU Exception in thread Thread-1 (run): Traceback (most recent call last): File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner self.run() File "/usr/lib/python3.10/threading.py", line 953, in run self._target(self._args, self._kwargs) File "/home/u/openvino-ai-plugins-gimp/gimpenv3/lib/python3.10/site-packages/gimpopenvino/plugins/stable-diffusion-ov/../../tools/stable-diffusion-ov-server.py", line 112, in run engine = StableDiffusionEngine( File "/home/u/openvino-ai-plugins-gimp/gimpenv3/lib/python3.10/site-packages/gimpopenvino/tools/openvino_common/models_ov/stable_diffusion_engine_NEW.py", line 110, in init self.text_encoder = self.core.compile_model(os.path.join(model, "text_encoder.xml"), device[0]) File "/home/u/openvino-ai-plugins-gimp/gimpenv3/lib/python3.10/site-packages/openvino/runtime/ie_api.py", line 399, in compile_model super().compile_model(model, device_name, {} if config is None else config), RuntimeError: Failed to create plugin /home/u/openvino-ai-plugins-gimp/gimpenv3/lib/python3.10/site-packages/openvino/libs/libopenvino_intel_gpu_plugin.so for device GPU Please, check your environment Check '!supported_devices.empty()' failed at src/plugins/intel_gpu/src/runtime/ocl/ocl_device_detector.cpp:212: [GPU] No supported OCL devices found or unexpected error happened during devices query. [GPU] Please check OpenVINO documentation for GPU drivers setup guide. `

[RyanMetcalfeInt8]

Hi @andyyeh75, would it be possible for you to run some standalone OpenVINO sample with GPU (for example, benchmark_app, hello_classification, etc.)? This will help determine whether something in the GIMP env is messed up, or if it's something related to system setup.

[gblong1]

Hi @andyyeh75 - Please install the latest OpenCL runtimes from: https://github.com/intel/compute-runtime/releases/tag/23.30.26918.9

Next, start the virtualenv for gimp by using source gimpenv3/bin/activate You can then open python, and see if you have the GPU as a device:

(gimpenv3) $ python3
Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from openvino.runtime import Core
>>> core = Core()
>>> core.get_available_devices()
['CPU', 'GPU']
>>>exit()

[andyyeh75]

Thanks @RyanMetcalfeInt8 and @gblong1 I have tested the way @gblong1 suggested but the log shows the same error.

For the python checks for core. before/after installed the new OCL runtime. GPU presents. The issue is we cannot load models by GPU. for your reference. (gimpenv3) u@u:~/openvino-ai-plugins-gimp$ python3 Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] on linux Type "help", "copyright", "credits" or "license" for more information.

from openvino.runtime import Core core = Core() File "", line 1 core = Core() IndentationError: unexpected indent core = Core() core.available_devices ['CPU', 'GPU']

Here is the a log which should be similar in the original bug.

Looking for GIMP process [ INFO ] Model Name: SD_1.5_square [ INFO ] Initializing Inference Engine... [ INFO ] Model Path: /home/u/openvino-ai-plugins-gimp/weights/stable-diffusion-ov/stable-diffusion-1.5/square [ INFO ] device_name: ['GPU', 'GPU', 'GPU'] Done looking for GIMP process gimp-2.99 process found: psutil.Process(pid=2, name='gimp-2.99', status='sleeping', started='10:16:54') Text Device: GPU Exception in thread Thread-1 (run): Traceback (most recent call last): File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner self.run() File "/usr/lib/python3.10/threading.py", line 953, in run self._target(*self._args, **self._kwargs) File "/home/u/openvino-ai-plugins-gimp/gimpenv3/lib/python3.10/site-packages/gimpopenvino/plugins/stable-diffusion-ov/../../tools/stable-diffusion-ov-server.py", line 112, in run engine = StableDiffusionEngine( File "/home/u/openvino-ai-plugins-gimp/gimpenv3/lib/python3.10/site-packages/gimpopenvino/tools/openvino_common/models_ov/stable_diffusion_engine_NEW.py", line 110, in init self.text_encoder = self.core.compile_model(os.path.join(model, "text_encoder.xml"), device[0]) File "/home/u/openvino-ai-plugins-gimp/gimpenv3/lib/python3.10/site-packages/openvino/runtime/ie_api.py", line 399, in compile_model super().compile_model(model, device_name, {} if config is None else config), RuntimeError: Failed to create plugin /home/u/openvino-ai-plugins-gimp/gimpenv3/lib/python3.10/site-packages/openvino/libs/libopenvino_intel_gpu_plugin.so for device GPU Please, check your environment Check '!supported_devices.empty()' failed at src/plugins/intel_gpu/src/runtime/ocl/ocl_device_detector.cpp:212: [GPU] No supported OCL devices found or unexpected error happened during devices query. [GPU] Please check OpenVINO documentation for GPU drivers setup guide.

[andyyeh75]

hi @RyanMetcalfeInt8

My thought is OBS plugin (https://github.com/intel/openvino-plugins-for-obs-studio can run normally with GPU as device. So I don't think it is a openvino environment issue but should be a gimp side of env issue.

[andyyeh75]

Tested with benchmark_app and shows GPU works normally as a device.

$benchmark_app -d GPU --path_to_model ./model_conv/lib/python3.10/site-packages/onnx/backend/test/data/pytorch-operator/test_operator_non_float_params/model.onnx [Step 1/11] Parsing and validating input arguments [ INFO ] Parsing input parameters [Step 2/11] Loading OpenVINO Runtime [ INFO ] OpenVINO: [ INFO ] Build ................................. 2022.3.0-9052-9752fafe8eb-releases/2022/3 [ INFO ] [ INFO ] Device info: [ INFO ] GPU [ INFO ] Build ................................. 2022.3.0-9052-9752fafe8eb-releases/2022/3 [ INFO ] [ INFO ] [Step 3/11] Setting device configuration [ WARNING ] Performance hint was not explicitly specified in command line. Device(GPU) performance hint will be set to THROUGHPUT. [Step 4/11] Reading model files [ INFO ] Loading model files [ INFO ] Read model took 1.54 ms [ INFO ] Original model I/O parameters: [ INFO ] Model inputs: [ INFO ] 0 (node: 0) : i64 / [...] / [2,2] [ INFO ] Model outputs: [ INFO ] 3 (node: 3) : i64 / [...] / [2,2] [Step 5/11] Resizing model to match image sizes and given batch [ INFO ] Model batch size: 1 [Step 6/11] Configuring input of the model [ INFO ] Model inputs: [ INFO ] 0 (node: 0) : i64 / [...] / [2,2] [ INFO ] Model outputs: [ INFO ] 3 (node: 3) : i64 / [...] / [2,2] [Step 7/11] Loading the model to the device [ INFO ] Compile model took 500.88 ms [Step 8/11] Querying optimal runtime parameters [ INFO ] Model: [ INFO ] NETWORK_NAME: torch-jit-export [ INFO ] OPTIMAL_NUMBER_OF_INFER_REQUESTS: 4 [ INFO ] PERF_COUNT: False [ INFO ] MODEL_PRIORITY: Priority.MEDIUM [ INFO ] GPU_HOST_TASK_PRIORITY: Priority.MEDIUM [ INFO ] GPU_QUEUE_PRIORITY: Priority.MEDIUM [ INFO ] GPU_QUEUE_THROTTLE: Priority.MEDIUM [ INFO ] GPU_ENABLE_LOOP_UNROLLING: True [ INFO ] CACHE_DIR: [ INFO ] PERFORMANCE_HINT: PerformanceMode.THROUGHPUT [ INFO ] COMPILATION_NUM_THREADS: 12 [ INFO ] NUM_STREAMS: 2 [ INFO ] PERFORMANCE_HINT_NUM_REQUESTS: 0 [ INFO ] INFERENCE_PRECISION_HINT: <Type: 'undefined'> [ INFO ] DEVICE_ID: 0 [Step 9/11] Creating infer requests and preparing input tensors [ WARNING ] No input files were given for input '0'!. This input will be filled with random values! [ INFO ] Fill input '0' with random values [Step 10/11] Measuring performance (Start inference asynchronously, 4 inference requests, limits: 60000 ms duration) [ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop). [ INFO ] First inference took 0.49 ms

[andyyeh75]

further log with $ clinfo. it looks quite normal to me. hope it helps. Thanks.

clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...) Intel(R) OpenCL Graphics clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...) Success [INTEL] clCreateContext(NULL, ...) [default] Success [INTEL] clCreateContextFromType(NULL, CL_DEVICE_TYPE_DEFAULT) Success (1) Platform Name Intel(R) OpenCL Graphics Device Name Intel(R) Graphics [0x46aa] clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU) No devices found in platform clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU) Success (1) Platform Name Intel(R) OpenCL Graphics Device Name Intel(R) Graphics [0x46aa] clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR) No devices found in platform clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM) No devices found in platform clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL) Success (1) Platform Name Intel(R) OpenCL Graphics Device Name Intel(R) Graphics [0x46aa]

ICD loader properties ICD loader Name OpenCL ICD Loader ICD loader Vendor OCL Icd free software ICD loader Version 2.2.14 ICD loader Profile OpenCL 3.0 clinfo_log.txt

[gblong1]

Hi @andyyeh75 - we have published a new release can you please update your code and see if you still see this issue? thanks!