search

intel / openvino-ai-plugins-gimp

GIMP AI plugins with OpenVINO Backend
Apache License 2.0
422 stars 47 forks source link

Couldn't find a suitable kernel when loading stable diffusion model #100

Closed TobKraus closed 3 months ago

TobKraus commented 3 months ago

Hello,

Thank you so much for this project! It helped me a lot lately. Now I completely re-setuped my PC, installed openvino, gimp & the plugins as described in the linux_install_guide (worked out well) and I run into this problem. It is notable that a few weeks ago on the very same hardware the openvino-ai-plugins-gimp worked perfectly (back then on Ubuntu 20.04, sadly I don't have any more specifications).

The error occurs whenever I want to load any of the stable diffusion models. Independent of the model or the specifications (where I am actually not able to specify which Text/Unet/VAE device I want to use, see screenshot - I don't know if this is a bug or just the current version of the plugin), GIMP crashes and the terminal output tells me "[GPU] Couldn't find a suitable kernel for convert". I've copied three examples of errors down below. They differ in the details but seem to all be caused by the same problem. I'm new to this whole topic and I really don't know how to handle this error. Maybe someone can help me? If you need further information from me about this, please tell me what to deliver. Screenshot from 2024-03-27 18-23-31

My system specifications:

+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.161.07             Driver Version: 535.161.07   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 3060 Ti     Off | 00000000:0A:00.0  On |                  N/A |
|  0%   45C    P0              39W / 220W |    780MiB /  8192MiB |      4%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

Three exemplary errors:

[ INFO ] Model Name: SD_1.5_Inpainting
[ INFO ] Initializing Inference Engine...
[ INFO ] Model Path: /home/kraus/openvino-ai-plugins-gimp/weights/stable-diffusion-ov/stable-diffusion-1.5-inpainting
Text Device: CPU
unet Device: GPU
Exception in thread Thread-1 (run):
Traceback (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/home/kraus/GIMP/gimpenv3/lib/python3.10/site-packages/gimpopenvino/plugins/stable-diffusion-ov/../../tools/stable-diffusion-ov-server.py", line 173, in run
    engine = StableDiffusionEngineInpainting(
  File "/home/kraus/GIMP/gimpenv3/lib/python3.10/site-packages/gimpopenvino/tools/openvino_common/models_ov/stable_diffusion_engine_inpainting.py", line 188, in __init__
    self.unet = self.core.compile_model(os.path.join(model, "unet.xml"), device[1]) #"unet_ov22_2.xml"
  File "/opt/intel/l_openvino_toolkit_ubuntu22_2024.0.0.14509.34caeefd078_x86_64/python/openvino/runtime/ie_api.py", line 515, in compile_model
    super().compile_model(model, device_name, {} if config is None else config),
RuntimeError: Exception from src/inference/src/cpp/core.cpp:123:
Exception from src/inference/src/dev/plugin.cpp:54:
Check 'false' failed at src/plugins/intel_gpu/src/plugin/program_builder.cpp:179:
[GPU] ProgramBuilder build failed!
Exception from src/plugins/intel_gpu/src/graph/include/primitive_type_base.h:58:
[GPU] Can't choose implementation for convert:Constant_47266 node (type=reorder)
[GPU] Original name: Constant_47266
[GPU] Original type: Convert
[GPU] Reason: Check '!kernels.empty()' failed at src/plugins/intel_gpu/src/kernel_selector/kernel_selector.cpp:70:
[GPU] Couldn't find a suitable kernel for convert:Constant_47266 params raw string: F16_BFYX_v1_p0_0_v1_p0_0_v320_p0_0_v320_p0_0;F32_BFYX_v1_p0_0_v1_p0_0_v320_p0_0_v320_p0_0

or:

[ INFO ] Model Name: SD_1.5_square_int8
[ INFO ] Initializing Inference Engine...
[ INFO ] Model Path: /home/kraus/openvino-ai-plugins-gimp/weights/stable-diffusion-ov/stable-diffusion-1.5/square_int8
[ INFO ] device_name: ['CPU', 'GPU', 'GPU', 'GPU']
[ DEBUG ] Starting new HTTPS connection (1): huggingface.co:443
[ DEBUG ] https://huggingface.co:443 "HEAD /openai/clip-vit-large-patch14/resolve/main/tokenizer_config.json HTTP/1.1" 200 0
Setting caching
Text Device: CPU
unet Device: GPU
unet-neg Device: GPU
2 warnings generated.
2 warnings generated.
    [...]
    [...] here are about 100 lines telling me about warnings generated
    [...]
76 warnings generated.
VAE Device: GPU
Exception in thread Thread-1 (run):
Traceback (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/home/kraus/GIMP/gimpenv3/lib/python3.10/site-packages/gimpopenvino/plugins/stable-diffusion-ov/../../tools/stable-diffusion-ov-server.py", line 142, in run
    engine = StableDiffusionEngineAdvanced(
  File "/home/kraus/GIMP/gimpenv3/lib/python3.10/site-packages/gimpopenvino/tools/openvino_common/models_ov/stable_diffusion_engine.py", line 157, in __init__
    self.vae_decoder = self.core.compile_model(os.path.join(model, "vae_decoder.xml"), device[3])
  File "/opt/intel/l_openvino_toolkit_ubuntu22_2024.0.0.14509.34caeefd078_x86_64/python/openvino/runtime/ie_api.py", line 515, in compile_model
    super().compile_model(model, device_name, {} if config is None else config),
RuntimeError: Exception from src/inference/src/cpp/core.cpp:123:
Exception from src/inference/src/dev/plugin.cpp:54:
Check 'false' failed at src/plugins/intel_gpu/src/plugin/program_builder.cpp:179:
[GPU] ProgramBuilder build failed!
Exception from src/plugins/intel_gpu/src/graph/include/primitive_type_base.h:58:
[GPU] Can't choose implementation for convert:Constant_19708 node (type=reorder)
[GPU] Original name: Constant_19708
[GPU] Original type: Convert
[GPU] Reason: Check '!kernels.empty()' failed at src/plugins/intel_gpu/src/kernel_selector/kernel_selector.cpp:70:
[GPU] Couldn't find a suitable kernel for convert:Constant_19708 params raw string: F16_BFYX_v1_p0_0_v1_p0_0_v512_p0_0_v512_p0_0;F32_BFYX_v1_p0_0_v1_p0_0_v512_p0_0_v512_p0_0

or:

[ INFO ] Model Name: SD_1.4
[ INFO ] Initializing Inference Engine...
[ INFO ] Model Path: /home/kraus/openvino-ai-plugins-gimp/weights/stable-diffusion-ov/stable-diffusion-1.4
[ DEBUG ] Starting new HTTPS connection (1): huggingface.co:443
[ DEBUG ] https://huggingface.co:443 "HEAD /openai/clip-vit-large-patch14/resolve/main/tokenizer_config.json HTTP/1.1" 200 0
Text Device: CPU
unet Device: GPU
Exception in thread Thread-1 (run):
Traceback (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/home/kraus/GIMP/gimpenv3/lib/python3.10/site-packages/gimpopenvino/plugins/stable-diffusion-ov/../../tools/stable-diffusion-ov-server.py", line 217, in run
    engine = StableDiffusionEngine(
  File "/home/kraus/GIMP/gimpenv3/lib/python3.10/site-packages/gimpopenvino/tools/openvino_common/models_ov/stable_diffusion_engine.py", line 514, in __init__
    self.unet = self.core.compile_model(os.path.join(model, "unet.xml"), device[1])  # "unet_ov22_2.xml"
  File "/opt/intel/l_openvino_toolkit_ubuntu22_2024.0.0.14509.34caeefd078_x86_64/python/openvino/runtime/ie_api.py", line 515, in compile_model
    super().compile_model(model, device_name, {} if config is None else config),
RuntimeError: Exception from src/inference/src/cpp/core.cpp:123:
Exception from src/inference/src/dev/plugin.cpp:54:
Check 'false' failed at src/plugins/intel_gpu/src/plugin/program_builder.cpp:179:
[GPU] ProgramBuilder build failed!
Exception from src/plugins/intel_gpu/src/graph/include/primitive_type_base.h:58:
[GPU] Can't choose implementation for convert:Constant_84954 node (type=reorder)
[GPU] Original name: Constant_84954
[GPU] Original type: Convert
[GPU] Reason: Check '!kernels.empty()' failed at src/plugins/intel_gpu/src/kernel_selector/kernel_selector.cpp:70:
[GPU] Couldn't find a suitable kernel for convert:Constant_84954 params raw string: F16_BFYX_v1_p0_0_v1_p0_0_v320_p0_0_v320_p0_0;F32_BFYX_v1_p0_0_v1_p0_0_v320_p0_0_v320_p0_0
gblong1 commented 3 months ago

Hi @TobKraus - I am glad you have found the project useful. However, we only officially support Intel based platforms. Also, the main branch is under development, and we are in the process of changing/updating the UI to simplify the options. If you would like to go back to the version which has the selection still available, please choose the 2.99-R1 release from the releases page.

TobKraus commented 3 months ago

Ah thank you very much. Sorry for overseeing the releases page, that was kind of a no-brainer if you think about it. With the 2.99-R1 version, I still get the same error when trying to run the model on my GPU, but I can just choose CPU for all the devices, then it works.