[Bug]: Compilation failed on iGPU for a custom clip model

arisha07 commented 4 days ago

OpenVINO Version

2024.4.0

Operating System

Windows System

Device used for inference

GPU

Framework

None

Model used

laion/CLIP-ViT-B-32-laion2B-s34B-b79K

Issue description

Compilation failed for a custom clip model on GPU. But, we are able to run benchmark_app on CPU & NPU on the same model. Will share the model IRs if requested.

Step-by-step reproduction

benchmark_app -m MeanCLIP.xml -niter 1 -d GPU

Relevant log output

benchmark_app -m MeanCLIP.xml -niter 1 -d GPU
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[Step 2/11] Loading OpenVINO Runtime
[ INFO ] OpenVINO:
[ INFO ] Build ................................. 2024.4.0-16579-c3152d32c9c-releases/2024/4
[ INFO ]
[ INFO ] Device info:
[ INFO ] GPU
[ INFO ] Build ................................. 2024.4.0-16579-c3152d32c9c-releases/2024/4
[ INFO ]
[ INFO ]
[Step 3/11] Setting device configuration
[ WARNING ] Performance hint was not explicitly specified in command line. Device(GPU) performance hint will be set to PerformanceMode.THROUGHPUT.
[Step 4/11] Reading model files
[ INFO ] Loading model files
[ INFO ] Read model took 29.91 ms
[ INFO ] Original model I/O parameters:
[ INFO ] Model inputs:
[ INFO ]     videos (node: videos) : f32 / [...] / [1,64,3,224,224]
[ INFO ] Model outputs:
[ INFO ]     embeddings (node: embeddings) : f32 / [...] / [1,512]
[Step 5/11] Resizing model to match image sizes and given batch
[ INFO ] Model batch size: 1
[Step 6/11] Configuring input of the model
[ INFO ] Model inputs:
[ INFO ]     videos (node: videos) : f32 / [...] / [1,64,3,224,224]
[ INFO ] Model outputs:
[ INFO ]     embeddings (node: embeddings) : f32 / [...] / [1,512]
[Step 7/11] Loading the model to the device
[ ERROR ] Exception from src/inference/src/cpp/core.cpp:107:
Exception from src/inference/src/dev/plugin.cpp:53:
Check 'false' failed at src/plugins/intel_gpu/src/plugin/program_builder.cpp:185:
[GPU] ProgramBuilder build failed!
Exception from src/plugins/intel_gpu/src/graph/include\primitive_type_base.h:59:
[GPU] Can't choose implementation for gemm:/visual/transformer/resblocks/resblocks.11/attn/Transpose_5 node (type=gemm)
[GPU] Original name: /visual/transformer/resblocks/resblocks.11/attn/Transpose_5
[GPU] Original type: Gemm
[GPU] Reason: could not create a primitive descriptor for a matmul primitive

Traceback (most recent call last):
  File "C:\Users\LNL\Projects\VisualRAG\GenAIExamples-client_inference\video_retrival\rag_win\lib\site-packages\openvino\tools\benchmark\main.py", line 408, in main
    compiled_model = benchmark.core.compile_model(model, benchmark.device, device_config)
  File "C:\Users\LNL\Projects\VisualRAG\GenAIExamples-client_inference\video_retrival\rag_win\lib\site-packages\openvino\runtime\ie_api.py", line 543, in compile_model
    super().compile_model(model, device_name, {} if config is None else config),
RuntimeError: Exception from src/inference/src/cpp/core.cpp:107:
Exception from src/inference/src/dev/plugin.cpp:53:
Check 'false' failed at src/plugins/intel_gpu/src/plugin/program_builder.cpp:185:
[GPU] ProgramBuilder build failed!
Exception from src/plugins/intel_gpu/src/graph/include\primitive_type_base.h:59:
[GPU] Can't choose implementation for gemm:/visual/transformer/resblocks/resblocks.11/attn/Transpose_5 node (type=gemm)
[GPU] Original name: /visual/transformer/resblocks/resblocks.11/attn/Transpose_5
[GPU] Original type: Gemm
[GPU] Reason: could not create a primitive descriptor for a matmul primitive

Issue submission checklist

[X] I'm reporting an issue. It's not a question.
[X] I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
[X] There is reproducer code and related data files such as images, videos, models, etc.

zulkifli-halim commented 3 days ago

Hi @arisha07

Can you share the model here for further investigation?

avitial commented 2 days ago

@arisha07 thanks for reaching out. Are you able to run other models on GPU successfully or do you observe also GPU issues with other models? Please also share the compute-runtime version you have installed on your system.

Ref. 155222

arisha07 commented 2 days ago

Yes, other models like Stable-Diffusion based models, Super-res, LLMs, video-llava etc. have no issues running on the GPU on this system.

Are you asking for the openvino version that I have installed ?

avitial commented 2 days ago

I apologize, I thought you were on Linux. What I meant was to share the graphics driver version you use.

arisha07 commented 2 days ago

I tried 5730 and 5736 GPU driver versions, both gave same errors.

openvinotoolkit / openvino