[Bug]: Failed to convert parameter to Python Representation when using benchmark_app on GPU in latency/throughput mode.

eugeooi commented 8 months ago

OpenVINO Version

2023.3.0

Operating System

Other (Please specify in description)

Device used for inference

GPU

Framework

None

Model used

omz_downloader --name efficientdet-d1-tf

Issue description

OS: Ubuntu 22.04 LTS Python: 3.10 CPU SKU: i7-1370PE OpenVINO: 2023.3.0

I have recently updated to OpenVINO version 2023.3.0, and run benchmark_app on efficientdet-d1-tf model from Open Model Zoo. No issue with CPUdevice, but error occurred when GPUdevice is selected.

Command used:

omz_downloader --name efficientdet-d1-tf
omz_converter --name efficientdet-d1-tf

benchmark_app -hint latency -m "efficientdet-d1-tf.xml" -d 'GPU'
benchmark_app -hint throughput -m "efficientdet-d1-tf.xml" -d 'GPU'

Output logs:

$ benchmark_app -hint latency -m "efficientdet-d1-tf.xml" -d 'GPU'
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[Step 2/11] Loading OpenVINO Runtime
[ INFO ] OpenVINO:
[ INFO ] Build ................................. 2023.3.0-13775-ceeafaf64f3-releases/2023/3
[ INFO ] 
[ INFO ] Device info:
[ INFO ] GPU
[ INFO ] Build ................................. 2023.3.0-13775-ceeafaf64f3-releases/2023/3
[ INFO ] 
[ INFO ] 
[Step 3/11] Setting device configuration
[Step 4/11] Reading model files
[ INFO ] Loading model files
[ INFO ] Read model took 31.41 ms
[ INFO ] Original model I/O parameters:
[ INFO ] Model inputs:
[ INFO ]     image_arrays:0 , image_arrays (node: image_arrays) : f32 / [N,H,W,C] / [1,640,640,3]
[ INFO ] Model outputs:
[ INFO ]     detections:0 , detections (node: detections) : f32 / [...] / [1,1,100,7]
[Step 5/11] Resizing model to match image sizes and given batch
[ INFO ] Model batch size: 1
[Step 6/11] Configuring input of the model
[ INFO ] Model inputs:
[ INFO ]     image_arrays:0 , image_arrays (node: image_arrays) : u8 / [N,H,W,C] / [1,640,640,3]
[ INFO ] Model outputs:
[ INFO ]     detections:0 , detections (node: detections) : f32 / [...] / [1,1,100,7]
[Step 7/11] Loading the model to the device
[ INFO ] Compile model took 762.22 ms
[Step 8/11] Querying optimal runtime parameters
[ INFO ] Model:
[ INFO ]   NETWORK_NAME: efficientdet-d1-tf
[ INFO ]   OPTIMAL_NUMBER_OF_INFER_REQUESTS: 1
[ INFO ]   PERF_COUNT: False
[ INFO ]   ENABLE_CPU_PINNING: False
[ INFO ]   MODEL_PRIORITY: Priority.MEDIUM
[ INFO ]   GPU_HOST_TASK_PRIORITY: Priority.MEDIUM
[ INFO ]   GPU_QUEUE_PRIORITY: Priority.MEDIUM
[ INFO ]   GPU_QUEUE_THROTTLE: Priority.MEDIUM
[ INFO ]   GPU_ENABLE_LOOP_UNROLLING: True
[ INFO ]   GPU_DISABLE_WINOGRAD_CONVOLUTION: False
[ INFO ]   CACHE_DIR: 
[ ERROR ] Unable to convert function return value to a Python type! The signature was
    (self: openvino._pyopenvino.CompiledModel, property: str) -> object
TypeError: Failed to convert parameter to Python representation!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/Desktop/env/openvino_env/lib/python3.10/site-packages/openvino/tools/benchmark/main.py", line 452, in main
    value = compiled_model.get_property(k)
TypeError: Unable to convert function return value to a Python type! The signature was
    (self: openvino._pyopenvino.CompiledModel, property: str) -> object

No issue when using OpenVINO 2023.2.0 to run the same benchmark_appcommand on GPU.

Step-by-step reproduction

No response

Relevant log output

No response

Issue submission checklist

[X] I'm reporting an issue. It's not a question.
[X] I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
[X] There is reproducer code and related data files such as images, videos, models, etc.

rkazants commented 8 months ago

@akuporos @jiwaszki, please take a look.

vladimir-paramuzov commented 8 months ago

The issue was already fixed by https://github.com/openvinotoolkit/openvino/pull/22094 So you may try some recent openvino-nightly package or build OV from sources. Alternatively, you may disable properties printout here in python benchmark_app to avoid this error

ilya-lavrenov commented 8 months ago

@eugeooi You can try to use latest openvino-nightly which contains this fix.

openvinotoolkit / openvino