Question about openvino_2020.1.033 and non AVX CPU

[Bleach665]

I use OpnVINO as OpenCV backend. Among the computers that use my software are quite old, but still powerful machines like 2x CPU Xeon x5660, 2x CPU Xeon e-2603, etc. with 64-180 Gb RAM. Unfortunately they not support avx instruction set, only sse4. Previous versions OpenVINO including 2019.3.379 allow use this old cpu on Windows, Linux and MacOS (I played with cpu_extension.* file and it works fine). In last OpenVINO release 2020.1.033 I receive error:

Failed to create plugin [my path to]\MKLDNNPlugin.dll for device CPU on demo executing. Also I dont find solution hove to rebuild some path of OpenVino lib (like cpu_extension.*) to use with sse cpu. This means that CPU without AVX instruction set will no longer be supported by OpenVINO inference engine?

[dkurt]

@Bleach665, Try to rebuild OpenVINO from source - CPU features should be detected automatically.

[yury-gorbachev]

Extension lib is no longer provided by OpenVINO binary and you don't need to provide it. All layers were moved to CPU (MKLDNN) plugin to increase out of the box coverage of the topologies. Can you describe setup that you have? e.g. do you continue using extension lib or just rely on distribution?

[Bleach665]

@dkurt. Previously, from time to time I tried to build Inference engine from sources, but there was always some error on one or several platforms (Win, Linux, Mac). Therefore, it was easier to use prebuilt lib from OpenVINO. Now I will probably do as you wrote - to build from sources. @yury-gorbachev. I just built demos, and use prebuilt lib from distribution. UPD. In previous version OpenVINO (<2020) to use with SSE CPU it was necessary to rename cpu_extension_avx2. to cpu_extension.. Cpu_extension was provided in ../OpenVINO/openvino/deployment_tools/inference_engine/bin/intel64/{Configuration} or was built from .../OpenVINO/openvino/deployment_tools/inference_engine/samples/ie_cpu_extension Nevertheless, look like that now easiest solution is fix issues and build inference engine from sources.

[yury-gorbachev]

@Bleach665 true, but I'm wondering why you're experiencing problems now when extension lib is not longer needed. Things should be simpler at least in theory.

[Bleach665]

In my case seems like the problem is that prebuilt lib in OpenVINO distribution are built with AXV baseline. Unfortunately I can't check this, because don't have a pc with a processor with AVX in touch.

[alalek]

Failed to create plugin [my path to]\MKLDNNPlugin.dll for device CPU

There are several lines in this error message. So you should dump all of them.

IMHO, This message has zero correlation with "cpu_extension" file. Check your environment settings (like message says), especially PATH variable.

{Configuration}

Start investigation from "Release" configuration. You should see the same "Release" in your [my path to]. Mess of Debug/Release would not work.

Additionally, you can investigate libraries loading attempts with help of Process Monitor.

---

This means that CPU without AVX instruction set will no longer be supported by OpenVINO inference engine

Check system requirements section - at least, Atom CPUs are still here.

[Bleach665]

There are several lines in this error message. So you should dump all of them.

Full command and output:

security_barrier_camera_demo.exe -i car_1.bmp -m "C:\Users\bleach\Documents\Intel\OpenVINO\openvino_models\ir\FP32\Security\object_detection\barrier\0106\dldt\vehicle-license-plate-detection-barrier-0106.xml" -m_va "C:\Users\bleach\Documents\Intel\OpenVINO\openvino_models\ir\FP32\Security\object_attributes\vehicle\resnet10_update_1\dldt\vehicle-attributes-recognition-barrier-0039.xml" -m_lpr "C:\Users\bleach\Documents\Intel\OpenVINO\openvino_models\ir\FP32\Security\optical_character_recognition\license_plate\dldt\license-plate-recognition-barrier-0001.xml" -d CPU -d_va CPU -d_lpr CPU
[ INFO ] InferenceEngine: 00007FFC273DB740
[ INFO ] Files were added: 1
[ INFO ]     car_1.bmp
[ INFO ] Loading device CPU
[ ERROR ] Failed to create plugin e:\Lib_prebuild\OpenVINO2020\TestIE2020\MKLDNNPlugin.dll for device CPU
Please, check your environment
Cannot load library 'e:\Lib_prebuild\OpenVINO2020\TestIE2020\MKLDNNPlugin.dll': 1114 from cwd: e:\Lib_prebuild\OpenVINO2020\TestIE2020

Check your environment settings (like message says), especially PATH variable.

For this test, I
-renamed the folder with my libraries where environment variable PATH point to -put demo exe, all Release dll, opencv dll (from OpenVINO distribution) tbb, ngraph to one folder to prevent build configuration and dll version mixture. The list of this directory:

```` cldnn_global_custom_kernels benchmark_app.exe cache.json car_1.bmp clDNNPlugin.dll compile_tool.exe crossroad_camera_demo.exe format_reader.dll format_reader.exp gaze_estimation_demo.exe gna.dll GNAPlugin.dll HDDLPlugin.dll hddl_perfcheck.exe HeteroPlugin.dll human_pose_estimation_demo.exe inference_engine.dll inference_engine_c_api.dll inference_engine_nn_builder.dll inference_engine_preproc.dll interactive_face_detection_demo.exe mask_rcnn_demo.exe MKLDNNPlugin.dll mkl_tiny_tbb.dll MultiDevicePlugin.dll multi_channel_face_detection_demo.exe multi_channel_human_pose_estimation_demo.exe multi_channel_object_detection_demo_yolov3.exe myriadPlugin.dll myriad_compile.exe myriad_perfcheck.exe ngraph.dll object_detection_demo_faster_rcnn.exe object_detection_demo_ssd_async.exe object_detection_demo_yolov3_async.exe opencv_calib3d420.dll opencv_core420.dll opencv_dnn420.dll opencv_features2d420.dll opencv_flann420.dll opencv_gapi420.dll opencv_highgui420.dll opencv_imgcodecs420.dll opencv_imgproc420.dll opencv_ml420.dll opencv_objdetect420.dll opencv_photo420.dll opencv_stitching420.dll opencv_version.exe opencv_versiond.exe opencv_version_win32.exe opencv_version_win32d.exe opencv_video420.dll opencv_videoio420.dll opencv_videoio_intel_mfx420_64.dll pcie-ma248x.elf pedestrian_tracker_demo.exe plugins.xml security_barrier_camera_demo.exe segmentation_demo.exe smart_classroom_demo.exe super_resolution_demo.exe tbb.dll text_detection_demo.exe usb-ma2450.mvcmd usb-ma2x8x.mvcmd ````

In the near future I will try to reproduce this on a clean virtual machine.

[alalek]

1114

It is GetLastError() result and it means ERROR_DLL_INIT_FAILED.

BTW, do you have installed some version of Intel compiler runtime? (setupvars should warn if it is missing)

[Bleach665]

Intel compiler runtime not installed now, but as far as I remember it was installed earlier. setupvars does not produce any warn:

setupvars.bat
Python 3.7.6
ECHO is off.
PYTHONPATH=e:\Lib_prebuild\OpenVINO2020\openvino_2020.1.033\deployment_tools\open_model_zoo\tools\accuracy_checker;e:\Lib_prebuild\OpenVINO2020\openvino_2020.1.033\python\python3.7;e:\Lib_prebuild\OpenVINO2020\openvino_2020.1.033\python\python3;e:\Lib_prebuild\OpenVINO2020\openvino_2020.1.033\deployment_tools\model_optimizer;
[setupvars.bat] OpenVINO environment initialized

[Bleach665]

Result of test on clean virtual machine: VBox used; OS windows10_x64_1909; host PC CPU - 2 x x5660 Xeon. Install:

• Python 3.6.5
• CMake 3.16.4
• Microsoft Visual Studio 2017 Community RTM 15.0.26228.4
• OpenVINO 2020.1.033_online.

Open console, run c:\Program Files (x86)\IntelSWTools\openvino\bin\setupvars.bat. After run demo_security_barrier_camera.bat from C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\demo

Final output:

"C:\Users\Bel\Documents\Intel\OpenVINO\inference_engine_demos_build\intel64\Release\security_barrier_camera_demo.exe" -i "c:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\demo\car_1.bmp"  -m "C:\Users\Bel\Documents\Intel\OpenVINO\openvino_models\ir\intel\vehicle-license-plate-detection-barrier-0106\FP16\vehicle-license-plate-detection-barrier-0106.xml" -m_lpr "C:\Users\Bel\Documents\Intel\OpenVINO\openvino_models\ir\intel\license-plate-recognition-barrier-0001\FP16\license-plate-recognition-barrier-0001.xml" -m_va "C:\Users\Bel\Documents\Intel\OpenVINO\openvino_models\ir\intel\vehicle-attributes-recognition-barrier-0039\FP16\vehicle-attributes-recognition-barrier-0039.xml" -d CPU -d_va CPU -d_lpr CPU
[ INFO ] InferenceEngine: 00007FFCBE98B740
[ INFO ] Files were added: 1
[ INFO ]     c:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\demo\car_1.bmp
[ INFO ] Loading device CPU
[ ERROR ] Failed to create plugin c:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\inference_engine\bin\intel64\Release\MKLDNNPlugin.dll for device CPU
Please, check your environment
Cannot load library 'c:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\inference_engine\bin\intel64\Release\MKLDNNPlugin.dll': 1114 from cwd: C:\Users\Bel\Documents\Intel\OpenVINO\inference_engine_demos_build\intel64\Release

Error

Also install_prerequisites.bat terminated with errors:

```` c:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\model_optimizer\install_prerequisites>install_prerequisites.bat Python 3.6.5 Режим вывода команд на экран (ECHO) отключен. Collecting tensorflow<2.0.0,>=1.2.0 (from -r ..\requirements.txt (line 1)) Using cached https://files.pythonhosted.org/packages/06/1f/3940c7bb51e1b9cf6e526c84d3239830c8d46c9823a3605945f9abb22411/tensorflow-1.15.2-cp36-cp36m-win_amd64.whl Collecting mxnet<=1.5.1,>=1.0.0 (from -r ..\requirements.txt (line 2)) Using cached https://files.pythonhosted.org/packages/a4/3e/07f6d61d8e0a5d50c6f128791a617d7b222224c52dacba04855a0b6ecfe6/mxnet-1.5.0-py2.py3-none-win_amd64.whl Collecting networkx>=1.11 (from -r ..\requirements.txt (line 3)) Using cached https://files.pythonhosted.org/packages/41/8f/dd6a8e85946def36e4f2c69c84219af0fa5e832b018c970e92f2ad337e45/networkx-2.4-py3-none-any.whl Requirement already satisfied: numpy>=1.12.0 in c:\users\bel\appdata\roaming\python\python36\site-packages (from -r ..\requirements.txt (line 4)) Requirement already satisfied: protobuf==3.6.1 in c:\users\bel\appdata\roaming\python\python36\site-packages (from -r ..\requirements.txt (line 5)) Collecting onnx>=1.1.2 (from -r ..\requirements.txt (line 6)) Using cached https://files.pythonhosted.org/packages/15/e4/0609d6a8463fcd246f64d1a65cc4fbc9e32f6357b4a6a13ae45c71cd01a3/onnx-1.6.0-cp36-cp36m-win_amd64.whl Collecting defusedxml>=0.5.0 (from -r ..\requirements.txt (line 7)) Using cached https://files.pythonhosted.org/packages/06/74/9b387472866358ebc08732de3da6dc48e44b0aacd2ddaa5cb85ab7e986a2/defusedxml-0.6.0-py2.py3-none-any.whl Collecting wrapt>=1.11.1 (from tensorflow<2.0.0,>=1.2.0->-r ..\requirements.txt (line 1)) Using cached https://files.pythonhosted.org/packages/ee/bc/7993faa8084b5a5dbabb07a197ae1b7590da4752dc80455d878573553e2f/wrapt-1.12.0.tar.gz Requirement already satisfied: tensorboard<1.16.0,>=1.15.0 in c:\users\bel\appdata\roaming\python\python36\site-packages (from tensorflow<2.0.0,>=1.2.0->-r ..\requirements.txt (line 1)) Collecting astor>=0.6.0 (from tensorflow<2.0.0,>=1.2.0->-r ..\requirements.txt (line 1)) Using cached https://files.pythonhosted.org/packages/c3/88/97eef84f48fa04fbd6750e62dcceafba6c63c81b7ac1420856c8dcc0a3f9/astor-0.8.1-py2.py3-none-any.whl Requirement already satisfied: gast==0.2.2 in c:\users\bel\appdata\roaming\python\python36\site-packages (from tensorflow<2.0.0,>=1.2.0->-r ..\requirements.txt (line 1)) Requirement already satisfied: grpcio>=1.8.6 in c:\users\bel\appdata\roaming\python\python36\site-packages (from tensorflow<2.0.0,>=1.2.0->-r ..\requirements.txt (line 1)) Collecting keras-preprocessing>=1.0.5 (from tensorflow<2.0.0,>=1.2.0->-r ..\requirements.txt (line 1)) Using cached https://files.pythonhosted.org/packages/28/6a/8c1f62c37212d9fc441a7e26736df51ce6f0e38455816445471f10da4f0a/Keras_Preprocessing-1.1.0-py2.py3-none-any.whl Requirement already satisfied: absl-py>=0.7.0 in c:\users\bel\appdata\roaming\python\python36\site-packages (from tensorflow<2.0.0,>=1.2.0->-r ..\requirements.txt (line 1)) Requirement already satisfied: opt-einsum>=2.3.2 in c:\users\bel\appdata\roaming\python\python36\site-packages (from tensorflow<2.0.0,>=1.2.0->-r ..\requirements.txt (line 1)) Requirement already satisfied: wheel>=0.26; python_version >= "3" in c:\users\bel\appdata\roaming\python\python36\site-packages (from tensorflow<2.0.0,>=1.2.0->-r ..\requirements.txt (line 1)) Requirement already satisfied: tensorflow-estimator==1.15.1 in c:\users\bel\appdata\roaming\python\python36\site-packages (from tensorflow<2.0.0,>=1.2.0->-r ..\requirements.txt (line 1)) Requirement already satisfied: termcolor>=1.1.0 in c:\users\bel\appdata\roaming\python\python36\site-packages (from tensorflow<2.0.0,>=1.2.0->-r ..\requirements.txt (line 1)) Requirement already satisfied: keras-applications>=1.0.8 in c:\users\bel\appdata\roaming\python\python36\site-packages (from tensorflow<2.0.0,>=1.2.0->-r ..\requirements.txt (line 1)) Requirement already satisfied: google-pasta>=0.1.6 in c:\users\bel\appdata\roaming\python\python36\site-packages (from tensorflow<2.0.0,>=1.2.0->-r ..\requirements.txt (line 1)) Requirement already satisfied: six>=1.10.0 in c:\users\bel\appdata\roaming\python\python36\site-packages (from tensorflow<2.0.0,>=1.2.0->-r ..\requirements.txt (line 1)) Collecting graphviz<0.9.0,>=0.8.1 (from mxnet<=1.5.1,>=1.0.0->-r ..\requirements.txt (line 2)) Using cached https://files.pythonhosted.org/packages/53/39/4ab213673844e0c004bed8a0781a0721a3f6bb23eb8854ee75c236428892/graphviz-0.8.4-py2.py3-none-any.whl Collecting requests<2.19.0,>=2.18.4 (from mxnet<=1.5.1,>=1.0.0->-r ..\requirements.txt (line 2)) Using cached https://files.pythonhosted.org/packages/49/df/50aa1999ab9bde74656c2919d9c0c085fd2b3775fd3eca826012bef76d8c/requests-2.18.4-py2.py3-none-any.whl Collecting decorator>=4.3.0 (from networkx>=1.11->-r ..\requirements.txt (line 3)) Using cached https://files.pythonhosted.org/packages/8f/b7/f329cfdc75f3d28d12c65980e4469e2fa373f1953f5df6e370e84ea2e875/decorator-4.4.1-py2.py3-none-any.whl Requirement already satisfied: setuptools in c:\users\bel\appdata\local\programs\python\python36\lib\site-packages (from protobuf==3.6.1->-r ..\requirements.txt (line 5)) Collecting typing-extensions>=3.6.2.1 (from onnx>=1.1.2->-r ..\requirements.txt (line 6)) Using cached https://files.pythonhosted.org/packages/03/92/705fe8aca27678e01bbdd7738173b8e7df0088a2202c80352f664630d638/typing_extensions-3.7.4.1-py3-none-any.whl Requirement already satisfied: werkzeug>=0.11.15 in c:\users\bel\appdata\roaming\python\python36\site-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow<2.0.0,>=1.2.0->-r ..\requirements.txt (line 1)) Requirement already satisfied: markdown>=2.6.8 in c:\users\bel\appdata\roaming\python\python36\site-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow<2.0.0,>=1.2.0->-r ..\requirements.txt (line 1)) Requirement already satisfied: h5py in c:\users\bel\appdata\roaming\python\python36\site-packages (from keras-applications>=1.0.8->tensorflow<2.0.0,>=1.2.0->-r ..\requirements.txt (line 1)) Requirement already satisfied: certifi>=2017.4.17 in c:\users\bel\appdata\roaming\python\python36\site-packages (from requests<2.19.0,>=2.18.4->mxnet<=1.5.1,>=1.0.0->-r ..\requirements.txt (line 2)) Requirement already satisfied: chardet<3.1.0,>=3.0.2 in c:\users\bel\appdata\roaming\python\python36\site-packages (from requests<2.19.0,>=2.18.4->mxnet<=1.5.1,>=1.0.0->-r ..\requirements.txt (line 2)) Collecting idna<2.7,>=2.5 (from requests<2.19.0,>=2.18.4->mxnet<=1.5.1,>=1.0.0->-r ..\requirements.txt (line 2)) Using cached https://files.pythonhosted.org/packages/27/cc/6dd9a3869f15c2edfab863b992838277279ce92663d334df9ecf5106f5c6/idna-2.6-py2.py3-none-any.whl Collecting urllib3<1.23,>=1.21.1 (from requests<2.19.0,>=2.18.4->mxnet<=1.5.1,>=1.0.0->-r ..\requirements.txt (line 2)) Using cached https://files.pythonhosted.org/packages/63/cb/6965947c13a94236f6d4b8223e21beb4d576dc72e8130bd7880f600839b8/urllib3-1.22-py2.py3-none-any.whl Building wheels for collected packages: wrapt Running setup.py bdist_wheel for wrapt ... error Failed building wheel for wrapt Running setup.py clean for wrapt Failed to build wrapt Installing collected packages: wrapt, astor, keras-preprocessing, tensorflow, graphviz, idna, urllib3, requests, mxnet, decorator, networkx, typing-extensions, onnx, defusedxml Running setup.py install for wrapt ... error Exception: Traceback (most recent call last): File "c:\users\bel\appdata\local\programs\python\python36\lib\site-packages\pip\compat\__init__.py", line 73, in console_to_str return s.decode(sys.__stdout__.encoding) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xaf in position 47: invalid start byte During handling of the above exception, another exception occurred: Traceback (most recent call last): File "c:\users\bel\appdata\local\programs\python\python36\lib\site-packages\pip\basecommand.py", line 215, in main status = self.run(options, args) File "c:\users\bel\appdata\local\programs\python\python36\lib\site-packages\pip\commands\install.py", line 342, in run prefix=options.prefix_path, File "c:\users\bel\appdata\local\programs\python\python36\lib\site-packages\pip\req\req_set.py", line 784, in install **kwargs File "c:\users\bel\appdata\local\programs\python\python36\lib\site-packages\pip\req\req_install.py", line 878, in install spinner=spinner, File "c:\users\bel\appdata\local\programs\python\python36\lib\site-packages\pip\utils\__init__.py", line 676, in call_subprocess line = console_to_str(proc.stdout.readline()) File "c:\users\bel\appdata\local\programs\python\python36\lib\site-packages\pip\compat\__init__.py", line 75, in console_to_str return s.decode('utf_8') UnicodeDecodeError: 'utf-8' codec can't decode byte 0xaf in position 47: invalid start byte You are using pip version 9.0.3, however version 20.0.2 is available. You should consider upgrading via the 'python -m pip install --upgrade pip' command. ***************************************************************************************** Warning: please expect that Model Optimizer conversion might be slow. You can boost conversion speed by installing protobuf-*.egg located in the "model-optimizer\install_prerequisites" folder or building protobuf library from sources. For more information please refer to Model Optimizer FAQ, question #80. ````

[alalek]

Thank you for double check and detailed steps!

I tested Win10 under QEMU/KVM with Westmere CPU emulation (without AVX/AVX2). I'm observing the same error message.

Windbg shows "illegal instruction" error with AVX command:

MKLDNNPlugin+0x9746d:
00007ffa`dbad746d c5fdefc0        vpxor   ymm0,ymm0,ymm0

``` ModLoad: 00007ffa`dba40000 00007ffa`dda0f000   C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\inference_engine\bin\intel64\Release\MKLDNNPlugin.dll ModLoad: 00007ffa`da9e0000 00007ffa`dba39000   C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\inference_engine\bin\intel64\Release\mkl_tiny_tbb.dll (410.146c): Illegal instruction - code c000001d (first chance) (410.146c): Illegal instruction - code c000001d (!!! second chance !!!) MKLDNNPlugin+0x9746d: 00007ffa`dbad746d c5fdefc0        vpxor   ymm0,ymm0,ymm0 0:000> k  # Child-SP          RetAddr           Call Site 00 00000045`088fbab0 00007ffb`0b54260e MKLDNNPlugin+0x9746d 01 00000045`088fbb10 00007ffa`dd18fdea ucrtbase!initterm+0x3e 02 00000045`088fbb40 00007ffa`dd18ff5c MKLDNNPlugin!CreatePluginEngine+0x150564a 03 00000045`088fbb70 00007ffb`0e171473 MKLDNNPlugin!CreatePluginEngine+0x15057bc 04 00000045`088fbbd0 00007ffb`0e1c6622 ntdll!LdrpCallInitRoutine+0x6f 05 00000045`088fbc40 00007ffb`0e1c646b ntdll!LdrpInitializeNode+0x15a 06 00000045`088fbd60 00007ffb`0e199e11 ntdll!LdrpInitializeGraphRecurse+0x73 07 00000045`088fbda0 00007ffb`0e16e399 ntdll!LdrpPrepareModuleForExecution+0xc5 08 00000045`088fbde0 00007ffb`0e171a67 ntdll!LdrpLoadDllInternal+0x19d 09 00000045`088fbe60 00007ffb`0e175b5c ntdll!LdrpLoadDll+0x107 0a 00000045`088fc000 00007ffb`0a6a2b6a ntdll!LdrLoadDll+0x8c 0b 00000045`088fc100 00007ffa`e619eec2 KERNELBASE!LoadLibraryExW+0x14a 0c 00000045`088fc170 00007ffa`e619e945 inference_engine!InferenceEngine::DeviceIDParser::DeviceIDParser+0x6252 0d 00000045`088fc7c0 00007ffa`e61995d4 inference_engine!InferenceEngine::DeviceIDParser::DeviceIDParser+0x5cd5 0e 00000045`088fc870 00007ffa`e6196f44 inference_engine!InferenceEngine::DeviceIDParser::DeviceIDParser+0x964 *** WARNING: Unable to verify checksum for C:\Users\admin\Documents\Intel\OpenVINO\inference_engine_demos_build\intel64\Release\security_barrier_camera_demo.exe 0f 00000045`088fdcd0 00007ff6`cd9e618a inference_engine!InferenceEngine::Core::GetVersions+0x434 10 00000045`088fdf90 00007ff6`cd9ff6b4 security_barrier_camera_demo+0x2618a 11 00000045`088ffcf0 00007ffb`0baf1fe4 security_barrier_camera_demo+0x3f6b4 12 00000045`088ffd30 00007ffb`0e1cef91 KERNEL32!BaseThreadInitThunk+0x14 13 00000045`088ffd60 00000000`00000000 ntdll!RtlUserThreadStart+0x21 0:000> u %rip MKLDNNPlugin+0x9746d: 00007ffa`dbad746d c5fdefc0        vpxor   ymm0,ymm0,ymm0 00007ffa`dbad7471 c5fc11055b26bc01 vmovups ymmword ptr [MKLDNNPlugin!CreatePluginEngine+0x1a0f334 (00007ffa`dd699ad4)],ymm0 00007ffa`dbad7479 48c7057026bc0100000000 mov qword ptr [MKLDNNPlugin!CreatePluginEngine+0x1a0f354 (00007ffa`dd699af4)],0 00007ffa`dbad7484 c5fc11057026bc01 vmovups ymmword ptr [MKLDNNPlugin!CreatePluginEngine+0x1a0f35c (00007ffa`dd699afc)],ymm0 00007ffa`dbad748c 48c7058526bc0100000000 mov qword ptr [MKLDNNPlugin!CreatePluginEngine+0x1a0f37c (00007ffa`dd699b1c)],0 00007ffa`dbad7497 c7058326bc0100000000 mov dword ptr [MKLDNNPlugin!CreatePluginEngine+0x1a0f384 (00007ffa`dd699b24)],0 00007ffa`dbad74a1 c5f9efc0        vpxor   xmm0,xmm0,xmm0 00007ffa`dbad74a5 c5f811442420    vmovups xmmword ptr [rsp+20h],xmm0 0:000> u %rip+0x38 MKLDNNPlugin+0x974a5: 00007ffa`dbad74a5 c5f811442420    vmovups xmmword ptr [rsp+20h],xmm0 00007ffa`dbad74ab 33c0            xor     eax,eax 00007ffa`dbad74ad 488d742420      lea     rsi,[rsp+20h] 00007ffa`dbad74b2 c5f877          vzeroupper 00007ffa`dbad74b5 0fa2            cpuid 00007ffa`dbad74b7 8906            mov     dword ptr [rsi],eax 00007ffa`dbad74b9 895e04          mov     dword ptr [rsi+4],ebx 00007ffa`dbad74bc 894e08          mov     dword ptr [rsi+8],ecx 0:000> r rax=0000000000000000 rbx=00000045088fb9e0 rcx=0000000000000001 rdx=000000000000101f rsi=00000045088fb4f0 rdi=00000045088fb4f0 rip=00007ffadbad746d rsp=00000045088fbab0 rbp=00000045088fbea4  r8=000001edf6025440  r9=000001edf6025620 r10=000001edf6025440 r11=000001edf60256d0 r12=00007ffadd19001c r13=0000000000000000 r14=00007ffadba40000 r15=0000000000000000 iopl=0         nv up ei pl zr na po nc cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010244 MKLDNNPlugin+0x9746d: 00007ffa`dbad746d c5fdefc0        vpxor   ymm0,ymm0,ymm0 ```

There are no debug symbols, so CreatePluginEngine symbol misleading here. MKLDNN plugin is crashed during initialization of global variables (initterm) before DllMain() call.

Looks like some code near cpuid instruction was compiled with incorrect flags.

---

Similar illegal instruction can be observed using Intel® Software Development Emulator (SDE) on any Windows machine (using -wsm platform parameter).

Looks like it is Windows-specific problem. Linux binaries passes SDE check.

/cc @ilya-lavrenov

---

I can't suggest work around to bypass this problem. Try to build and use DLDT from sources from here: https://github.com/opencv/dldt/tree/2020.1 (see build-instruction.md file)

[w3sip]

Just want to chime in -- seeing the same behavior. Previous releases (2019) work on a i5-3470 machine, while 2020R1 fails (but does work on newer CPUs). From the library loading perspective, I see both versions look for (and fail to find) libmalloc.dll, but in case of success lookup of tbbmalloc.dll is then successfully performed. In case of failure it never happens.

[w3sip]

Any updates on this, guys? Is there a workaround, or waiting for the next release is the only way (and will the next release fix this)?

[mosdav]

@Bleach665 I'm also facing the very same issue, linking against the debug libraries (MKLDNNPlugin**d**.dll, the likes) work for me (kind of W/A)

[Bleach665]

UPD. Everything written below was tested on a CPU without AVX support. Same issue with DLDT built from source. DLDT rev. a347375d0188b58056887f382161c6ed6fc3dcb3, Windows 10, CMake GUI 3.17.0, VS 2017 15.9.21. As @mosdav wrote, with debug build all work fine, both DLDT sample app/bench and IE as OpenCV backend. In release build we have errors, same as described above: benchmark_app:

e:\Lib_prebuild\DLDT\dldt\bin\intel64\Release>benchmark_app.exe -i e:\q\lena.jpg -m e:\Lib_prebuild\DLDT\dldt\model-optimizer\alexnet.xml -d CPU -api sync
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[ INFO ] Files were added: 1
[ INFO ]     e:\q\lena.jpg
[ WARNING ] -nstreams default value is determined automatically for a device. Although the automatic selection usually provides a reasonable performance,but it still may be non-optimal for some cases, for more information look at README.

[Step 2/11] Loading Inference Engine
[ INFO ] InferenceEngine:
        API version ............ 2.1
        Build .................. custom_2020_a347375d0188b58056887f382161c6ed6fc3dcb3
        Description ....... API
[ INFO ] Device info:
[ ERROR ] Failed to create plugin e:\Lib_prebuild\DLDT\dldt\bin\intel64\Release\MKLDNNPlugin.dll for device CPU
Please, check your environment
Cannot load library 'e:\Lib_prebuild\DLDT\dldt\bin\intel64\Release\MKLDNNPlugin.dll': 1114 from cwd: e:\Lib_prebuild\DLDT\dldt\bin\intel64\Release

built IE as backend in Opencv:

OpenCV(4.3.0-pre) E:\Lib_prebuild\opencv\source\opencv\modules\dnn\src\op_inf_engine.cpp:881: error: (-2:Unspecified error) in function 'cv::dnn::InfEngineBackendNet::initPlugin'
> Failed to initialize Inference Engine backend (device = CPU): Failed to create plugin c:\lib\path\MKLDNNPlugin.dll for device CPU
> Please, check your environment
> Cannot load library 'c:\lib\path\MKLDNNPlugin.dll': 1114 from cwd: C:\00CurWork\ImProcQCV\ImProcQCV
> 

[alalek]

Code with problem looks like this.

@Bleach665 To workaround that you can try to comment out these lines:

• REG_FACTORY_FOR_TYPE(avx512_common, ...
• REG_FACTORY_FOR_TYPE(avx2, ...
• do not touch lines with "sse" or "isa_any" to avoid performance regressions.

These files should be processed: argmax.cpp proposal.cpp resample.cpp

This is not a proper fix but should allow to run application on non-AVX CPU.

Problem is that cross-compiled dispatched code (mkldnn_plugin_layers_avx2 and mkldnn_plugin_layers_avx512 targets) has global dynamic initializers. These "dispatched" binary files should not have global dynamic initializers at all:

• their code is called unconditionally during library (DLL) load
• compiler generates initializers code using AVX instructions (due cross-compilation with /arch:AVX2 or similar flags).

How to detect / check that problem is gone without running program

1. We need `dumpbin` + `argmax.obj` file from a build directory `mkldnn_plugin_layers_avx2.dir\Release\` (AVX2 (or AVX512) here is important). 2. Run command: `dumpbin.exe /ALL /OUT:argmax.log .\argmax.obj` 3. Search for [`.CRT$XCU`](https://docs.microsoft.com/en-us/cpp/c-runtime-library/crt-initialization?view=vs-2019) section in `argmax.log` file. You should see something like this: ``` SECTION HEADER #94A .CRT$XCU name 0 physical address 0 virtual address 10 size of raw data EF4E4 file pointer to raw data (000EF4E4 to 000EF4F3) EF4F4 file pointer to relocation table 0 file pointer to line numbers 2 number of relocations 0 number of line numbers 40400040 flags Initialized Data 8 byte align Read Only RAW DATA #94A 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ RELOCATIONS #94A Symbol Symbol Offset Type Applied To Index Name -------- ---------------- ----------------- -------- ------ 00000000 ADDR64 00000000 00000000 B96 ??__Ecpu@?A0x35d88e89@cpu@impl@mkldnn@@YAXXZ (void __cdecl mkldnn::impl::cpu::`anonymous namespace'::`dynamic initializer for 'cpu''(void)) 00000008 ADDR64 00000000 00000000 BBC ??__E__reg__ArgMax@Cpu@Extensions@InferenceEngine@@YAXXZ (void __cdecl InferenceEngine::Extensions::Cpu::`dynamic initializer for '__reg__ArgMax''(void)) ``` This **INVALID** dispatched binary have 2 pointers on code with dynamic initializers. This code runs unconditionally during DLL load. **Fixed binary** should have zero size of `.CRT$XCU` section or the section should gone from the binary and its dump.

[Bleach665]

@alalek, thanks. With this changes IE work fine on non AVX CPU. I tested it on the benchmark_app and IE as backend in OpenCV. However, after fix, .CRT$XCU section in argmax.obj has no zero size. Before fix:

.CRT$XCU name
       0 physical address
       0 virtual address
      10 size of raw data
   324E2 file pointer to raw data (000324E2 to 000324F1)
   324F2 file pointer to relocation table
       0 file pointer to line numbers
       2 number of relocations
       0 number of line numbers
40400040 flags
         Initialized Data
         8 byte align
         Read Only

After:

.CRT$XCU name
       0 physical address
       0 virtual address
       8 size of raw data
    11AE file pointer to raw data (000011AE to 000011B5)
    11B6 file pointer to relocation table
       0 file pointer to line numbers
       1 number of relocations
       0 number of line numbers
40400040 flags
         Initialized Data
         8 byte align
         Read Only

@yury-gorbachev, @w3sip, @mosdav, it might interest you.

[JulienMaille]

Is this issue still present with just released 2020.2?

[alalek]

This issue is not fixed in 2020.2

[andeyeluguo]

My cpu core is i7 8750, still has this problem, anybody can help?

[dkurt]

@andeyeluguo, not sure that current issue is your case. This is about non-AVX configurations.

[AJV009]

Hi guys am a newbie programmer in OpenVINO environment. I went through the whole thread I couldn't figure out what should be done to run the latest OpenVINO toollkit on my i5-3320M (HP EliteBook 8470p) Could someone write a script or some sort of program for people for like to me to use the Toolkit on older CPU version OR should I download OpenVINO 2019?

[brmarkus]

@AJV009 can you describe your environment, please? Do you use MS-WIN or Linux or MacOS? I use OpenVINO 2020.1 also on an Intel ATOM "ApolloLake", which doesn't have AVX CPU instructions. I'm mainly using OpenVINO in a Docker container - as it installs quite a lot which clashes with some of my packages on my HOST. OpenVINO comes with a lot of samples and demos - many of them could be used with Python and no need for cmake/make compilation. However, many samples&demos are C or C++ and need to be compiled. Which steps have you tried to far, where are you stuck?

In this GitHub repo there are several threads about AVX and CPU-Extension. You might find additional information there.

[AJV009]

Okay I use MS-WIN with Anaconda as my package/project manager. somewhere here -> https://docs.openvinotoolkit.org/2020.2/_docs_install_guides_installing_openvino_windows.html#model_optimizer_configuration_steps Should I really reproduce the errors? (let me try again!) I faced a TON of warnings which I ignored, but then ended up with a error while running the samples mentioned in the installation guide. (so I thought maybe my CPU wasn't supported, cause they already mentioned about CPUs >= 6th gen)

I did a clean install this time it worked don't know what happened before! BUT yeah I faced a new error -

###############|| Run Inference Engine classification sample ||###############
Waiting for 1 seconds, press a key to continue ...
        1 file(s) copied.
classification_sample_async.exe -i "C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\demo\car.png" -m "C:\Users\XeonAJ\Documents\Intel\OpenVINO\openvino_models\ir\public\squeezenet1.1\FP16\squeezenet1.1.xml" -d CPU
[ INFO ] InferenceEngine:
        API version ............ 2.1
        Build .................. 42025
        Description ....... API
[ INFO ] Parsing input parameters
[ INFO ] Parsing input parameters
[ INFO ] Files were added: 1
[ INFO ]     C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\demo\car.png
[ INFO ] Creating Inference Engine
[ ERROR ] Failed to create plugin C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\inference_engine\bin\intel64\Release\MKLDNNPlugin.dll for device CPU
Please, check your environment
Cannot load library 'C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\inference_engine\bin\intel64\Release\MKLDNNPlugin.dll': 1114 from cwd: C:\Users\XeonAJ\Documents\Intel\OpenVINO\inference_engine_samples_build\intel64\Release
Error

Is there any quick fixes that I should do?

[ilya-lavrenov]

Update: the issue addressed in the latest master. Could you please try?

Now, we have a special cmake macro to safely compile sources for different ISAs (see https://github.com/openvinotoolkit/openvino/blob/master/inference-engine/src/mkldnn_plugin/CMakeLists.txt#L172-L187)

CC @AlexPeskov

[JulienMaille]

@ilya-lavrenov Did this fix missed the 2020.3 release? I see that the known issues mentions

28747 | CPU plugin does not work on Windows system with CPUs less then AVX2 instruction set (Intel Atom® processors) Manually rebuild the CPU plugin from sources available in the public repository with CMake feature flags ENABLE_AVX2=OFF and ENABLE_AVX512=OFF.

[ilya-lavrenov]

The fix is big enough and it was decided (by @dmitry-gorokhov @AlexPeskov) not to port it to 2020.3 release.

[w3sip]

So when is the 2020.4 scheduled for? Considering it's a regression from 2019 releases, it's unfortunate to see it pushed back.

[w3sip]

As a suggestion -- maybe it's worth having default distribution shipping all flavors of the CPU plugin; the client app could then evaluate CPU caps, and generate plugins.xml to load the suitable version.

[ilya-lavrenov]

@w3sip you can build CPU plugin and Inference Engine libraries from sources using current master branch. Does this way work for you? This check from your side would be very helpful.

[w3sip]

Unfortunately, our current CI workflow is to ingest the Intel versioned binary distribution. Switch to building from source is under consideration at the moment (and definitely best avoided, if possible). If we do go that route, are there any guidelines regarding the toolchain? Versions of clang/cmake, etc?

[asuhov]

@w3sip Build toolchains for OpenVINO are described here: https://github.com/openvinotoolkit/openvino/blob/master/build-instruction.md

[Bleach665]

@ilya-lavrenov. You may be interested in this. I built OpenVINO from branch releases/2020/4, rev ae9e0510f008220a297130c45e30bfc7fcf27b04 and it work fine on non-AVX CPU. I tested on classification_sample_async, object_detection_sample_ssd and IE as OpenCV backend.

[ilya-lavrenov]

@Bleach665 thanks! Does it means that the original issue has gone?

[Bleach665]

@ilya-lavrenov As you see fit. But I think it's better to wait for the release OpenVINO v.2020.4, test it, and if it works well at non-AVX CPU, then close this issue.

[JulienMaille]

@w3sip Build toolchains for OpenVINO are described here: https://github.com/openvinotoolkit/openvino/blob/master/build-instruction.md

I'm about to try to compile it on my Windows machine. Instructions mentions an extra download: Intel Graphics Drivers https://github.com/openvinotoolkit/openvino/blob/master/build-instruction.md#software-requirements-1 However my build machine does not have an Intel GPU. Do you confirm this driver is not required for compilation, only for execution of the GPU plugin?

[Bleach665]

Do you confirm this driver is not required for compilation, only for execution of the GPU plugin?

Absolutely right, driver is not required for compilation.

[JulienMaille]

@Bleach665 Can you confirm this is solved in 2020.4? Thanks a lot

[Bleach665]

@JulienMaille Unfortunately, I can't confirm this now - I don't have a suitable PC on hand. I can check this only in a few days.

[smbunn]

I have just installed openvino 2020.4 on a Windows 10 computer with a Ryzen 7 1700 CPU and 32 GB of RAM. That CPU suports AVX and AVX2. It failed on the two demo's with the same error as above i.e.

[ ERROR ] Failed to create plugin MKLDNNPlugin.dll for device CPU

My three previous versions all worked (2019.1.087, 2019.2.242, 2019.3.334) and I had installed 2020.2.117 but never tested it.

[AlexPeskov]

@ilya-lavrenov. You may be interested in this. I built OpenVINO from branch releases/2020/4, rev ae9e051 and it work fine on non-AVX CPU. I tested on classification_sample_async, object_detection_sample_ssd and IE as OpenCV backend.

The GitHub master contains corresponding fix since 20/05/2020 update. So all inherited branches (2020.4 as well) have no such issue. And your local result confirms that.

I've just check it with 2020.4 binaries with Intel SDE emulation tool. The crash is gone.

[Bleach665]

I just finished testing. OpenVINO 2020.4 work perfect on non-AVX CPU. Win10, VS 2017; OpenCV build with OpenVINO 2020.4, TBB and MKL. Tested on security_barrier_camera_demo and IE as OpenCV backend.

[Abdalnablse10]

Hey, I have a intel atom x5 z8500 cpu, I use debian and i keep getting "illegal instruction" after building whisper.cpp using openvino 2023.0.0, i tried building openvino from source with -DENABLE_AVX512F=OFF and -DENABLE_AVX2=OFF but i kept getting "illegal instruction".