[Bug] get_property reports wrong performance hint when model is compiled in AUTO THROUGHPUT mode

EdjeElectronics commented 1 year ago

System information (version)

OpenVINO=> 2022.2
Operating System / Platform => Windows 11 64 Bit
Compiler => Python runtime installed using PyPI
Problem classification => n/a

Detailed description

When a model is compiled with device="AUTO" and config={"PERFORMANCE_HINT":"THROUGHPUT"}, the get_property method reports that the model is using PerformanceMode.LATENCY, which is incorrect. This is using the Python runtime installed through PyPI.

Steps to reproduce

The problem can be reproduced by running the following Python code. When the code below is run, it will reporty PERFORMANCE_HINT: PerformanceMode.LATENCY even though the model has been compiled in THROUGHPUT mode. I also uploaded a notebook (forked from openvino_notebooks) that can be used to reproduce the problem: https://github.com/EdjeElectronics/openvino_notebooks/blob/main/notebooks/108-query-device/108-query-device.ipynb

# Initialize OpenVINO
from openvino.runtime import Core
ie = Core()

# Download model
base_model_dir = "model"
model_name = "person-detection-0202"
precision = "FP16"
model_path = (
    f"model/intel/{model_name}/{precision}/{model_name}.xml"
)
download_command = f"omz_downloader " \
                   f"--name {model_name} " \
                   f"--precision {precision} " \
                   f"--output_dir {base_model_dir} " \
                   f"--cache_dir {base_model_dir}"
! $download_command

# Compile model in AUTO mode with THROUGHPUT performance hint
model = ie.read_model(model=model_path)
compiled_model = ie.compile_model(model, "AUTO", {"PERFORMANCE_HINT":"THROUGHPUT"})

# Check PERFORMANCE_HINT property
hint_property = compiled_model.get_property("PERFORMANCE_HINT")
print('PERFORMANCE_HINT: %s' % hint_property)

Issue submission checklist

[X] I report the issue, it's not a question
[X] I checked the problem with documentation, FAQ, open issues, Stack Overflow, etc and have not found solution
[X] There is reproducer code and related data files: images, videos, models, etc.

cc @helena-intel @ryanloney

Iffa-Intel commented 1 year ago

@EdjeElectronics perhaps you could try to declare the config as a variable (as in here), by doing this, you'll be definite with the config since you could call the variable and confirms:

May I know which Python version did you use for this? since I'm getting errors for your custom code (your hint_property algorithm):

EdjeElectronics commented 1 year ago

Hi @Iffa-Meah , thanks for looking at this. I tried declaring the config as a variable, and I got the same result. Here is the code I used:

# Compile model in AUTO mode with THROUGHPUT performance hint
model = ie.read_model(model=model_path)
config = {"PERFORMANCE_HINT":"THROUGHPUT"}
compiled_model = ie.compile_model(model, "AUTO", config)

# Check PERFORMANCE_HINT property
hint_property = compiled_model.get_property("PERFORMANCE_HINT")
print('PERFORMANCE_HINT: ', hint_property)

Result: PERFORMANCE_HINT: PerformanceMode.LATENCY

I am using Python 3.7.13. Sorry my print code didn't work, I changed it to just use print('PERFORMANCE_HINT: ', hint_property). That should work!

Iffa-Intel commented 1 year ago

@EdjeElectronics I noticed the same behaviour:

This seems suspicious. We'll further investigate this and get back to you asap with the workaround.

EdjeElectronics commented 1 year ago

Hi @Iffa-Meah , are there any updates on this?

peterchen-intel commented 1 year ago

@EdjeElectronics This was fixed for 2022.3, you can try with master or 2022.3 preview release https://github.com/openvinotoolkit/openvino/releases/tag/2022.3.0.dev20221125

EdjeElectronics commented 1 year ago

Thanks, I tested it with 2022.3.0.dev202211225 and it works as expected now. The issue has been fixed.

hbalasu1 commented 1 year ago

Good to hear that @EdjeElectronics. With that, I will close this case. Thank you

openvinotoolkit / openvino