[Bug]: OpenVINO crashes (SIGILL) when enumerating core.available_devices on J4105 and older

koush commented 1 week ago

OpenVINO Version

2024.3.0

Operating System

Other (Please specify in description)

Device used for inference

CPU

Framework

None

Model used

All Models

Issue description

OS: Ubuntu 22.04 LXC inside Proxmox.

OpenVINO crashes on J4105 and older (and possibly some generation newer) CPU. This issue started in 2024.3.0. Last good build was 2024.2.0 I can resolve the issue by simply downgrading the package.

~~Running this same code on the Proxmox host shell (instead of LXC) works fine. This may be a containerization issue.~~ See comment below where I've diagnosed the root cause.

Step-by-step reproduction

Repro code:

import openvino.runtime as ov

core = ov.Core()
print("Available devices:")
for device in core.available_devices:
    print(f"{device} :")

Relevant log output

(.venv) root@scrypted-wyse:/tmp/work# python3 test.py 
Available devices:
Illegal instruction

Issue submission checklist

[X] I'm reporting an issue. It's not a question.
[X] I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
[x] There is reproducer code and related data files such as images, videos, models, etc.

koush commented 1 week ago

The issue actually seems to stem from the apt package level-zero being installed. If I remove it, it no longer crashes. While this is not an issue with bare metal installation, it is an issue with creating docker images or lxc templates. The image can't contain the NPU support or risk crashing non-NPU systems.

level-zero is assuming that certain instructions exist while it is being probed.

rkazants commented 1 week ago

@ilya-lavrenov, could you please advice on this issue?

RyanMetcalfeInt8 commented 1 week ago

Just to add -- this issue has been reported several times for our updated set of of OpenVINO-based plugins for Audacity, which recently upgraded to use 2024.3. Here are a couple of examples:

https://forum.audacityteam.org/t/how-to-fix-unable-to-load-the-mod-openvino-module-error/116996/

https://github.com/intel/openvino-plugins-ai-audacity/issues/295

https://forum.audacityteam.org/t/cant-install-openvino-correctly-need-help-with-installation/118089/2

The workaround for these folks is to delete the NPU plugin DLL.

Edit: Note that the above reported issues have been on Windows, but seems to be the same issue (illegal instruction).

openvinotoolkit / openvino