search

intel / linux-npu-driver

Intel® NPU (Neural Processing Unit) Driver
MIT License
189 stars 18 forks source link

NPU fails with OpenVINO 2024.3 "Unsupported operation aten::max_pool2d/MaxPool with type MaxPool" (still broken with OpenVINO 2024.4) #44

Closed mattcurf closed 5 days ago

mattcurf commented 3 months ago

NPU is failing with OpenVINO 2024.3 + NPU driver 1.6.0 using Resnet50 v1.5 model and OpenVINO benchmark app. Worked previously with OpenVINO 2024.2 and NPU driver 1.2. ASUS NUC 14 Pro, Intel(R) Core(TM) Ultra 5 125H

matt@matt-nuc-mtl:~/openvino_play$ sudo dmesg | grep vpu
[    3.626154] intel_vpu 0000:00:0b.0: enabling device (0000 -> 0002)
[    3.638500] intel_vpu 0000:00:0b.0: [drm] Firmware: intel/vpu/vpu_37xx_v0.0.bin, version: 20240726*MTL_CLIENT_SILICON-release*0004*ci_tag_ud202428_vpu_rc_20240726_0004*e4a99ed6b3e
[    3.732504] [drm] Initialized intel_vpu 1.0.0 20230117 for 0000:00:0b.0 on minor 0

root@f2d20769c6c1:/project# dpkg -l | grep npu
ii  intel-driver-compiler-npu    1.6.0.20240814-10390978568              amd64        Intel(R) Driver Compiler for NPU hardware
ii  intel-level-zero-npu         1.6.0.20240814-10390978568              amd64        Intel(R) Level Zero Driver for NPU hardware

root@c0b4c9315308:/project# /root/openvino_cpp_samples_build/intel64/Release/benchmark_app -m model/ir_model/resnet50_fp16.xml -d NPU -hint throughput -niter 1000
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[Step 2/11] Loading OpenVINO Runtime
[ INFO ] OpenVINO:
[ INFO ] Build ................................. 2024.3.0-16041-1e3b88e4e3f-releases/2024/3
[ INFO ] 
[ INFO ] Device info:
[ INFO ] NPU
[ INFO ] Build ................................. 2024.3.0-16041-1e3b88e4e3f-releases/2024/3
[ INFO ] 
[ INFO ] 
[Step 3/11] Setting device configuration
[Step 4/11] Reading model files
[ INFO ] Loading model files
[ INFO ] Read model took 4.85 ms
[ INFO ] Original model I/O parameters:
[ INFO ] Network inputs:
[ INFO ]     x (node: x) : f32 / [...] / [1,3,224,224]
[ INFO ] Network outputs:
[ INFO ]     x.45 (node: aten::linear/Add) : f32 / [...] / [1,1000]
[Step 5/11] Resizing model to match image sizes and given batch
[ WARNING ] x: layout is not set explicitly, so it is defaulted to NCHW. It is STRONGLY recommended to set layout manually to avoid further issues.
[Step 6/11] Configuring input of the model
[ INFO ] Model batch size: 1
[ INFO ] Network inputs:
[ INFO ]     x (node: x) : u8 / [N,C,H,W] / [1,3,224,224]
[ INFO ] Network outputs:
[ INFO ]     x.45 (node: aten::linear/Add) : f32 / [...] / [1,1000]
[Step 7/11] Loading the model to the device
[ ERROR ] Exception from src/inference/src/cpp/core.cpp:104:
Exception from src/inference/src/dev/plugin.cpp:53:
Exception from src/plugins/intel_npu/src/plugin/src/plugin.cpp:674:
Exception from src/plugins/intel_npu/src/plugin/src/compiled_model.cpp:61:
Check 'result == ZE_RESULT_SUCCESS' failed at src/plugins/intel_npu/src/compiler/src/zero_compiler_in_driver.cpp:807:
Failed to compile network. L0 createGraph result: ZE_RESULT_ERROR_UNKNOWN, code 0x7ffffffe. [NOT IMPLEMENTED] Unsupported operation aten::max_pool2d/MaxPool with type MaxPool. Try to update the driver to the latest version. If the error persists, please submit a bug report in https://github.com/openvinotoolkit/openvino/issues
Failed to create executable
mattcurf commented 3 months ago

Full repro case at https://github.com/mattcurf/openvino_play, model of origin via torchvision at https://github.com/mattcurf/openvino_play/blob/master/download_resnet50.py

dmatveev commented 3 months ago

The notebook has been updated recently with an older model, please pull the latest OV notebooks repo and try again.

mattcurf commented 3 months ago

I do not use OV notebooks, and the mentioned workaround is not a solution for enabling NPU support with common torchvision based models.

frenchwr commented 2 months ago

Hello, I just noticed this thread. Maybe not a permanent solution but I encountered the same thing a few weeks back and the second suggested workaround here worked for me.

mattcurf commented 2 months ago

Still appears broken with latest OpenVINO 2024.4 and NPU driver 1.8 but with slightly different error message

matt@matt-nuc-mtl:~/openvino_play$  sudo dmesg | grep vpu
[    3.535580] intel_vpu 0000:00:0b.0: enabling device (0000 -> 0002)
[    3.553824] intel_vpu 0000:00:0b.0: [drm] Firmware: intel/vpu/vpu_37xx_v0.0.bin, version: 20240820*MTL_CLIENT_SILICON-release*1902*ci_tag_ud202436_vpu_rc_20240820_1902*a4634b5107c
[    3.645105] [drm] Initialized intel_vpu 1.0.0 20230117 for 0000:00:0b.0 on minor 0

root@f79e5ec806b9:/project# dpkg -l | grep npu
ii  intel-driver-compiler-npu    1.8.0.20240916-10885588273              amd64        Intel(R) Driver Compiler for NPU hardware
ii  intel-level-zero-npu         1.8.0.20240916-10885588273              amd64        Intel(R) Level Zero Driver for NPU hardware

(ov) root@f79e5ec806b9:/project# /root/openvino_cpp_samples_build/intel64/Release/benchmark_app -m model/ir_model/resnet50_fp16.xml -d NPU -hint throughput -niter 1000
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[Step 2/11] Loading OpenVINO Runtime
[ INFO ] OpenVINO:
[ INFO ] Build ................................. 2024.4.0-16579-c3152d32c9c-releases/2024/4
[ INFO ] 
[ INFO ] Device info:
[ INFO ] NPU
[ INFO ] Build ................................. 2024.4.0-16579-c3152d32c9c-releases/2024/4
[ INFO ] 
[ INFO ] 
[Step 3/11] Setting device configuration
[Step 4/11] Reading model files
[ INFO ] Loading model files
[ INFO ] Read model took 4.92 ms
[ INFO ] Original model I/O parameters:
[ INFO ] Network inputs:
[ INFO ]     x (node: x) : f32 / [...] / [1,3,224,224]
[ INFO ] Network outputs:
[ INFO ]     x.45 (node: aten::linear/Add) : f32 / [...] / [1,1000]
[Step 5/11] Resizing model to match image sizes and given batch
[ WARNING ] x: layout is not set explicitly, so it is defaulted to NCHW. It is STRONGLY recommended to set layout manually to avoid further issues.
[Step 6/11] Configuring input of the model
[ INFO ] Model batch size: 1
[ INFO ] Network inputs:
[ INFO ]     x (node: x) : u8 / [N,C,H,W] / [1,3,224,224]
[ INFO ] Network outputs:
[ INFO ]     x.45 (node: aten::linear/Add) : f32 / [...] / [1,1000]
[Step 7/11] Loading the model to the device
[ ERROR ] Exception from src/inference/src/cpp/core.cpp:104:
Exception from src/inference/src/dev/plugin.cpp:53:
Exception from src/plugins/intel_npu/src/plugin/src/plugin.cpp:697:
Exception from src/plugins/intel_npu/src/plugin/src/compiled_model.cpp:62:
Exception from src/plugins/intel_npu/src/compiler/src/zero_compiler_in_driver.cpp:848:
L0 pfnCreate2 result: ZE_RESULT_ERROR_UNKNOWN, code 0x7ffffffe
mattcurf commented 2 months ago

Hello, I just noticed this thread. Maybe not a permanent solution but I encountered the same thing a few weeks back and the second suggested workaround here worked for me.

I've confirmed the mentioned workaround #2 works for me, changing opset14 to opset8

mattcurf commented 5 days ago

This appears fixed now with combination of OpenVINO 2024.5 and latest NPU driver :) https://github.com/mattcurf/openvino_play/pull/5