Is Core threading safe ?

[Jsy0220]

Hi, I tried to use OpenVino in my application for multiple model inference. I have some question about ov::Core:

1. Does it needed one for each model in one application ?
2. If only single one needed, Are all methods of it thread-safe ?

[ilya-lavrenov]

Hi @Jsy0220

You can have a single Core for your application. The responsibility of Core is to manage OpenVINO inference devices (plugins) and main methods are:

• compile_model
• some manipulation with devices to get their properties (like get_property)
• read_model

These methods are thread safe and if you access the same device from multiple threads (eg. call compile_model for different models) or read different models in parallel (via read_model) - the Core will ensure proper initialization of inference plugins and frontends to read models.

[Jsy0220]

@ilya-lavrenov I see. Thank you!!

[Jsy0220]

@ilya-lavrenov Hi, I tried to use one core in my application, but it seems failed. My application includes multiple models and each one runs in a thread and owns a core, that is ok. Now I modified to all models share one core and each one still runs in a thread respectively, but one of the models would load failed at ov::core::compile_model() due to Cannot create reorder primitive: unsupported reorder case. Do you know why ?

openvino version: 2022.1.0

[ilya-lavrenov]

it's not related to single / multiple cores usage or single / multiple threads.

Could you please try latest 2024.5 OpenVINO? You are using 2.5 year old release

Is it possible to share your model to reproduce on our side?

[Jsy0220]

@ilya-lavrenov sorry, I can't share this model. Do you know some possible reasons ？It only happened under single core. Do models share some resource if using one core ? And another question: is there any advantage using single core in one application compare to using multiple cores ？ Thank you!!