Open joey5678 opened 2 weeks ago
@mlyashko, @rzubarev, please help to resolve it.
Any update ?
BTW, I got another error when running benchmark_app with INT8 yolov8 model under the same test environment:
error: MultiClusterStrategyAssignment Pass failed : Cannot get per cluster memory shapes. Unsupported distribution: #VPU.DistributedTensor<mode = <SEGMENTED>, num_tiles = [1, 1, 2, 1], num_clusters = 2 : i64, alignment = [1, 1, 4, 1]>
completed output log:
./benchmark_app -d NPU -m ~/share/models/public/yolo-v8n/INT8/yolo-v8n.xml -t 15
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[Step 2/11] Loading OpenVINO Runtime
[ INFO ] OpenVINO:
[ INFO ] Build ................................. 2024.1.0-15008-f4afc983258-releases/2024/1
[ INFO ]
[ INFO ] Device info:
[ INFO ] NPU
[ INFO ] Build ................................. 2024.1.0-15008-f4afc983258-releases/2024/1
[ INFO ]
[ INFO ]
[Step 3/11] Setting device configuration
[ WARNING ] Performance hint was not explicitly specified in command line. Device(NPU) performance hint will be set to THROUGHPUT.
[Step 4/11] Reading model files
[ INFO ] Loading model files
[ INFO ] Read model took 11.67 ms
[ INFO ] Original model I/O parameters:
[ INFO ] Network inputs:
[ INFO ] images (node: images) : f32 / [...] / [1,3,640,640]
[ INFO ] Network outputs:
[ INFO ] output0 (node: output0) : f32 / [...] / [1,84,8400]
[ INFO ] onnx::Reshape_421 (node: onnx::Reshape_421) : f32 / [...] / [1,144,80,80]
[ INFO ] onnx::Reshape_436 (node: onnx::Reshape_436) : f32 / [...] / [1,144,40,40]
[ INFO ] onnx::Reshape_451 (node: onnx::Reshape_451) : f32 / [...] / [1,144,20,20]
[Step 5/11] Resizing model to match image sizes and given batch
[ WARNING ] images: layout is not set explicitly, so it is defaulted to NCHW. It is STRONGLY recommended to set layout manually to avoid further issues.
[Step 6/11] Configuring input of the model
[ INFO ] Model batch size: 1
[ INFO ] Network inputs:
[ INFO ] images (node: images) : u8 / [N,C,H,W] / [1,3,640,640]
[ INFO ] Network outputs:
[ INFO ] output0 (node: output0) : f32 / [...] / [1,84,8400]
[ INFO ] onnx::Reshape_421 (node: onnx::Reshape_421) : f32 / [...] / [1,144,80,80]
[ INFO ] onnx::Reshape_436 (node: onnx::Reshape_436) : f32 / [...] / [1,144,40,40]
[ INFO ] onnx::Reshape_451 (node: onnx::Reshape_451) : f32 / [...] / [1,144,20,20]
[Step 7/11] Loading the model to the device
error: MultiClusterStrategyAssignment Pass failed : Cannot get per cluster memory shapes. Unsupported distribution: #VPU.DistributedTensor<mode = <SEGMENTED>, num_tiles = [1, 1, 2, 1], num_clusters = 2 : i64, alignment = [1, 1, 4, 1]>
[ ERROR ] Exception from src/inference/src/cpp/core.cpp:106:
Exception from src/inference/src/dev/plugin.cpp:54:
Exception from src/plugins/intel_npu/src/plugin/src/plugin.cpp:513:
Check 'result == ZE_RESULT_SUCCESS' failed at src/plugins/intel_npu/src/compiler/src/zero_compiler_in_driver.cpp:745:
Failed to compile network. L0 createGraph result: ZE_RESULT_ERROR_UNKNOWN, code 0x7ffffffe. Compilation failed
Failed to create executable
If with FP16 yolov8 model, it's OK.
attached the INT8 yolov8 model I use. my-yolo-v8n-int8.zip
OpenVINO Version
2024.1.0-15008-f4afc983258-releases/2024/1
Operating System
Other (Please specify in description)
Device used for inference
NPU
Framework
TensorFlow 1
Model used
yolo-v4-tf
Issue description
When running command:
./benchmark_app -d NPU -m /home/aibox/share/models/public/yolo-v4-tf/FP16/yolo-v4-tf.xml -t 20
Got error: [ ERROR ] Exception from src/plugins/intel_npu/src/backend/include/zero_utils.hpp:21: L0 zeFenceHostSynchronize result: ZE_RESULT_ERROR_DEVICE_LOST, code 0x70000001 - device hung, reset, was removed, or driver update occurred
Same error if with yolo-v4-tf INT8 model
Step-by-step reproduction
Setup Ubuntu22.04.3 OS system on MTL platform (Ultra 7 165HL)
Install GPU driver and NPU v1.2 driver Install OpenVINO 2024.1 Run install_dependency script Build CPP benchmark_app
Relevant log output
Issue submission checklist