search

openvinotoolkit / openvino

OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference
https://docs.openvino.ai
Apache License 2.0
7.29k stars 2.27k forks source link

[Build]: Does not build on musl libc #24368

Closed blacklightpy closed 6 months ago

blacklightpy commented 6 months ago

OpenVINO Version

Master Branch

Operating System

Other (Please specify in description)

Hardware Architecture

x86 (64 bits)

Target Platform

jyothish@void ~> lscpu && lscpu -e
Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         39 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  4
  On-line CPU(s) list:   0-3
Vendor ID:               GenuineIntel
  Model name:            Intel(R) Pentium(R) CPU 4417U @ 2.30GHz
    CPU family:          6
    Model:               142
    Thread(s) per core:  2
    Core(s) per socket:  2
    Socket(s):           1
    Stepping:            10
    CPU(s) scaling MHz:  100%
    CPU max MHz:         2300.0000
    CPU min MHz:         400.0000
    BogoMIPS:            4599.93
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm p
                         be syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aper
                         fmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popc
                         nt tsc_deadline_timer aes xsave rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb pti ssbd ibrs ibpb stibp tpr_shadow
                         flexpriority ept vpid ept_ad fsgsbase tsc_adjust smep erms invpcid mpx rdseed smap clflushopt intel_pt xsaveopt xsave
                         c xgetbv1 xsaves dtherm arat pln pts hwp hwp_notify hwp_act_window hwp_epp vnmi md_clear flush_l1d arch_capabilities
Virtualization features:
  Virtualization:        VT-x
Caches (sum of all):
  L1d:                   64 KiB (2 instances)
  L1i:                   64 KiB (2 instances)
  L2:                    512 KiB (2 instances)
  L3:                    2 MiB (1 instance)
NUMA:
  NUMA node(s):          1
  NUMA node0 CPU(s):     0-3
Vulnerabilities:
  Gather data sampling:  Not affected
  Itlb multihit:         KVM: Mitigation: VMX disabled
  L1tf:                  Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable
  Mds:                   Mitigation; Clear CPU buffers; SMT vulnerable
  Meltdown:              Mitigation; PTI
  Mmio stale data:       Mitigation; Clear CPU buffers; SMT vulnerable
  Retbleed:              Mitigation; IBRS
  Spec rstack overflow:  Not affected
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:            Mitigation; IBRS, IBPB conditional, STIBP conditional, RSB filling, PBRSB-eIBRS Not affected
  Srbds:                 Mitigation; Microcode
  Tsx async abort:       Not affected
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE    MAXMHZ   MINMHZ       MHZ
  0    0      0    0 0:0:0:0          yes 2300.0000 400.0000 2299.9980
  1    0      0    1 1:1:1:0          yes 2300.0000 400.0000 2299.9900
  2    0      0    0 0:0:0:0          yes 2300.0000 400.0000 2300.0049
  3    0      0    1 1:1:1:0          yes 2300.0000 400.0000 2300.0020

Build issue description

Code doesn't build on musl, as it is hardcoded to GLIBC at many points.

I'm using Void Linux musl libc edition.

musl libc is also used in Alpine Linux. pipx install openvino isn't working for me either, as it says no matching distribution found or something of that sort. I guess I'll use the container image in the meantime.

Here are the references:

openvino (master)> grep -inr OV_GLIBC

CMakeLists.txt:89:if(NOT OV_GLIBC_VERSION VERSION_EQUAL 0.0)
CMakeLists.txt:90:    message (STATUS "GLIBC_VERSION ......................... " ${OV_GLIBC_VERSION})
cmake/dependencies.cmake:104:    elseif(LINUX AND X86_64 AND OV_GLIBC_VERSION VERSION_GREATER_EQUAL 2.17)
cmake/dependencies.cmake:135:    elseif(LINUX AND AARCH64 AND OV_GLIBC_VERSION VERSION_GREATER_EQUAL 2.17)
cmake/developer_package/target_flags.cmake:155:function(ov_glibc_version)
cmake/developer_package/target_flags.cmake:158:        ov_get_compiler_definition("__GLIBC__" _ov_glibc_major)
cmake/developer_package/target_flags.cmake:159:        ov_get_compiler_definition("__GLIBC_MINOR__" _ov_glibc_minor)
cmake/developer_package/target_flags.cmake:161:        set(OV_GLIBC_VERSION "${_ov_glibc_major}.${_ov_glibc_minor}" PARENT_SCOPE)
cmake/developer_package/target_flags.cmake:163:        set(OV_GLIBC_VERSION "0.0" PARENT_SCOPE)
cmake/developer_package/target_flags.cmake:167:ov_glibc_version()
cmake/developer_package/target_flags.cmake:174:        ov_get_compiler_definition("_GLIBCXX_USE_CXX11_ABI" OV_GLIBCXX_USE_CXX11_ABI)
src/bindings/python/wheel/CMakeLists.txt:61:    string(REPLACE "." "_" _ov_glibc_version "${OV_GLIBC_VERSION}")
src/bindings/python/wheel/CMakeLists.txt:62:    set(manylinux "manylinux_${_ov_glibc_version}")
src/cmake/install_tbb.cmake:94:    if(OV_GLIBC_VERSION VERSION_LESS_EQUAL 2.26)
src/cmake/openvino.cmake:56:if(DEFINED OV_GLIBCXX_USE_CXX11_ABI)
src/cmake/openvino.cmake:57:    target_compile_definitions(${TARGET_NAME} PUBLIC _GLIBCXX_USE_CXX11_ABI=${OV_GLIBCXX_USE_CXX11_ABI})
src/plugins/proxy/src/plugin.cpp:27:#        define OV_GLIBC_VERSION_LESS_2_34
src/plugins/proxy/src/plugin.cpp:599:#ifdef OV_GLIBC_VERSION_LESS_2_34
src/plugins/proxy/src/plugin.cpp:613:#ifdef OV_GLIBC_VERSION_LESS_2_34

Build script or step-by-step to reproduce

Do CMake. It won't configure.

Relevant log output

> cmake -DCMAKE_BUILD_TYPE=Release -DENABLE_SYSTEM_TBB=ON -DENABLE_AVX2=OFF -DENABLE_AVX512F=OFF ..
CMake Error at cmake/developer_package/target_flags.cmake:145 (message):
  Failed to detect '__GLIBC__' definition value:

Call Stack (most recent call first):
  cmake/developer_package/target_flags.cmake:158 (ov_get_compiler_definition)
  cmake/developer_package/target_flags.cmake:167 (ov_glibc_version)
  cmake/developer_package/features.cmake:6 (include)
  cmake/developer_package/OpenVINODeveloperScriptsConfig.cmake:46 (include)
  CMakeLists.txt:46 (find_package)

-- Configuring incomplete, errors occurred!

Issue submission checklist

wenjiew commented 6 months ago

I don't think this Linux distribution and musl lib are being supported. @ilya-lavrenov to offer more insights. Thanks!

blacklightpy commented 6 months ago

The condition here is that the project lists only Linux as a dependency, but actually also has a dependency on GLIBC, which is an implementation of the system C API or libc.

What you refer to as Linux distributions are actually GNU/Linux style system distributions, where Linux is only the kernel.

GNU was developed at a time when there were no free UNIX operating systems like BSD. By the time they had completed a POSIX userspace, Linus Torvalds had created the Linux kernel, and the GNU team added compatibility with Linux as its kernel, and that's how the idea of GNU/Linux distributions was made (in contrast to complete OS projects like BSD or Android).

So GNU/Linux is a stand in for any kind of OS that consists of a POSIX userland paired with the Linux kernel (pretty much in the same manner as JCB, Kleenex or Band-Aid). Most distros like Ubuntu and RHEL use the GNU glibc, while there are others that use musl libc.

The name Linux is more widely used to describe these systems because people found GNU/Linux weird to pronounce, and many people also hated the Richard Stallman, the founder of GNU and FSF, and did not want to associate with it.

Many distributions like Ubuntu and RHEL also use systemd software suite as the Initialization System and Service Manager, which often comes as a hard dependency, but OpenVINO is less likely to have dependencies on it, as it is a low-level framework.

Perhaps compatibility with musl should be added as a patch downstream, but it can certainly be helpful if any hard dependencies can be removed. Alpine Linux OCI image is popularly used as a base for containers because it's lightweight, due to its use of musl libc and Busybox system utilities.

ilya-lavrenov commented 6 months ago

@blacklightpy is it possible to use some docker image on typical glib-based Linux (let's say Ubuntu 22.04) to reproduce the issue with musl libc?

blacklightpy commented 6 months ago

I'll try that.

ilya-lavrenov commented 6 months ago

glibc is not mandatory for OpenVINO. I think you can add detection of musl libc version in a similar way and add second path to OpenVINO cmake scripts.

blacklightpy commented 6 months ago

In the container, I get 

/home/openvino/build# cmake --build . --parallel -v
Change Dir: '/home/openvino/build'

Run Build Command(s): /usr/bin/cmake -E env VERBOSE=1 /usr/bin/gmake -f Makefile -j
gmake: Makefile: No such file or directory
gmake: *** No rule to make target 'Makefile'.  Stop.

My CMake configure command was cmake -DCMAKE_BUILD_TYPE=Release -DENABLE_SYSTEM_TBB=ON -DENABLE_AVX2=OFF -DENABLE_AVX512F=OFF .. and that phase worked.

EDIT: The CMake configure was incomplete, let me see.

EDIT: Fixed it, somehow the files in the plugins/intel_cpu/thirdparty/mlas submodule directory was deleted, and that caused an error.

ilya-lavrenov commented 6 months ago

which docker image do you use?

blacklightpy commented 6 months ago

Ubuntu 24.04.

ilya-lavrenov commented 6 months ago

How can I use musl libc in Ubuntu 24.04?

blacklightpy commented 6 months ago

You cannot use musl directly in Ubuntu because it provides the libc.so and other important libraries. You'll have to use an Alpine Linux or Void Linux container.

Void Linux: ghcr.io/void-linux/void-musl Alpine Linux: alpine

EDIT: I'll get the dependency lists needed for XBPS (package manager) if you're testing Void.

blacklightpy commented 6 months ago

The dependencies are: sudo xbps-install ca-certificates file base-devel ninja scons ccache cmake pkgconf git shellcheck patchelf fdupes gzip tbb-devel pugixml-devel ocl-icd-devel OpenCL-Headers rapidjson libva-devel snappy-devel python3-pip python3-virtualenv python3-setuptools python3-devel python3-pybind11 libffi-devel python3-enchant wget flatbuffers-devel git-lfs enchant2-devel json-c++

EDIT: sudo xbps-install -Sy would be sudo apt update (should run before installing), and sudo xbps-install -u would be sudo apt upgrade (might need, being a rolling release distro).

blacklightpy commented 6 months ago

Also you should use enter with /bin/dash to use the shell, because it doesn't come with bash preinstalled. (Quirks: you cannot press up arrow to go back in command history in dash)

And for clear command, you'll need to install ncurses.

blacklightpy commented 6 months ago

Also my build inside the Ubuntu container failed like:

gmake[2]: Leaving directory '/home/openvino/build'
[  1%] Built target opencv_c_wrapper
/usr/bin/ranlib /home/openvino/bin/intel64/Release/libmlas.a
gmake[2]: Leaving directory '/home/openvino/build'
[  1%] Built target mlas
gmake[2]: Leaving directory '/home/openvino/build'
gmake[1]: *** [CMakeFiles/Makefile2:2988: thirdparty/gflags/gflags/CMakeFiles/gflags_nothreads_static.dir/all] Error 2
gmake[2]: Leaving directory '/home/openvino/build'
gmake[1]: *** [CMakeFiles/Makefile2:3014: thirdparty/protobuf/protobuf/cmake/CMakeFiles/libprotobuf-lite.dir/all] Error 2
gmake[2]: Leaving directory '/home/openvino/build'
gmake[1]: *** [CMakeFiles/Makefile2:5815: src/plugins/intel_npu/thirdparty/level-zero/source/CMakeFiles/ze_loader.dir/all] Error 2
gmake[1]: Leaving directory '/home/openvino/build'
gmake: *** [Makefile:156: all] Error 2

But I don't think this has anything to do with musl, since it links to glibc.

ilya-lavrenov commented 6 months ago

The dependencies are: sudo xbps-install ca-certificates file base-devel ninja scons ccache cmake pkgconf git shellcheck patchelf fdupes gzip tbb-devel pugixml-devel ocl-icd-devel OpenCL-Headers rapidjson libva-devel snappy-devel python3-pip python3-virtualenv python3-setuptools python3-devel python3-pybind11 libffi-devel python3-enchant wget flatbuffers-devel git-lfs enchant2-devel json-c++

EDIT: sudo xbps-install -Sy would be sudo apt update (should run before installing), and sudo xbps-install -u would be sudo apt upgrade (might need, being a rolling release distro).

Could you please support of these dependencies to https://github.com/openvinotoolkit/openvino/blob/master/install_build_dependencies.sh ? Based on it, I will try to compile OpenVINO.

ilya-lavrenov commented 6 months ago

@blacklightpy I've made initial support in PR https://github.com/openvinotoolkit/openvino/pull/24428 Could you please try?

blacklightpy commented 6 months ago

@ilya-lavrenov it passed configuration stage and started building.

blacklightpy commented 6 months ago

@ilya-lavrenov building in parallel crashed by compositor, and building in TTY also would not let me do anything else. I'll have to retry with just 2 cores, but it'll probably take some time. Meanwhile I've added the dependency lists to your fork in my PR as requested.

ilya-lavrenov commented 6 months ago

@ilya-lavrenov building in parallel crashed by compositor, and building in TTY also would not let me do anything else. I'll have to retry with just 2 cores, but it'll probably take some time. Meanwhile I've added the dependency lists to your fork in my PR as requested.

I've merged your PR, thank you. Do you think it's also worth to add branch for Alpine Linux? As I see, it uses different package manager and we would like to have support for it. Could you please try adding it?

blacklightpy commented 6 months ago

Sure, lemme see. I like tracking down package names.

blacklightpy commented 6 months ago

I saw that you added OpenCL-CLHPP, python3-pip and python3-wheel. Missing out python3-pip was a mistake on my part, but I didn't see python3-wheel required in the other distros' dependency lists.

Anyways, I've added it to Alpine Linux's dependency list too.

ilya-lavrenov commented 6 months ago

@blacklightpy thank you. Could you please send PR directly to OpenVINO official repo? My PR was merged to master

blacklightpy commented 6 months ago

Alright