Cannot build Python bindings or with Intel GPU support

[nec4]

Hello. The tools look very interesting but I am having trouble building them properly for a user-level, non-root installation:

$ mkdir build
$ cd build
$ export level_zero_DIR=$HOME/.local/include/level_zero
$ cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$HOME/.local -DPYBIND11_FINDPYTHON=ON ..
-- The CXX compiler identification is GNU 13.2.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /sw/compiler/gcc/13.2.0/a100/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Setting the hardware sampler interval to 100ms.
-- Enable sampling of CPU information using lscpu.
-- Enable sampling of CPU information using free.
-- Can't use turbostat without root permissions! Try using with sudo...
-- Can't use turbostat with root if a sudo password is required! Please add turbostat to the sudoer group.
-- Disabling turbostat support!
-- Couldn't find package subprocess.h. Building version b6e1611d430e3019c423d2af26bb162e7ed5c3ae from source.
-- Hardware sampling for NVIDIA GPUs disabled!
-- Hardware sampling for AMD GPUs disabled!
-- Hardware sampling for Intel GPUs disabled!
-- Building Python language bindings for PLSSVM.
-- Found Python: /home/bzfbnick/miniforge3/envs/perun/bin/python3.9 (found version "3.9.19") found components: Interpreter Development Development.Module Development.Embed
-- Couldn't find package pybind11. Building version master from source.
-- pybind11 v2.14.0 dev1
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- Configuring done (10.9s)
-- Generating done (0.1s)
-- Build files have been written to: /home/bzfbnick/local/hardware_sampling/build
$ make
[  8%] Building CXX object CMakeFiles/hardware_sampling.dir/src/hardware_sampling/event.cpp.o
[ 16%] Building CXX object CMakeFiles/hardware_sampling.dir/src/hardware_sampling/hardware_sampler.cpp.o
[ 25%] Building CXX object CMakeFiles/hardware_sampling.dir/src/hardware_sampling/utility.cpp.o
[ 33%] Building CXX object CMakeFiles/hardware_sampling.dir/src/hardware_sampling/cpu/hardware_sampler.cpp.o
[ 41%] Building CXX object CMakeFiles/hardware_sampling.dir/src/hardware_sampling/cpu/cpu_samples.cpp.o
[ 50%] Building CXX object CMakeFiles/hardware_sampling.dir/src/hardware_sampling/cpu/utility.cpp.o
[ 58%] Linking CXX shared library libhardware_sampling.so
[ 58%] Built target hardware_sampling
[ 66%] Building CXX object bindings/CMakeFiles/HardwareSampling.dir/event.cpp.o
[ 75%] Building CXX object bindings/CMakeFiles/HardwareSampling.dir/hardware_sampler.cpp.o
[ 83%] Building CXX object bindings/CMakeFiles/HardwareSampling.dir/main.cpp.o
[ 91%] Building CXX object bindings/CMakeFiles/HardwareSampling.dir/cpu_hardware_sampler.cpp.o
[100%] Linking CXX shared module HardwareSampling.cpython-39-x86_64-linux-gnu.so
lto-wrapper: warning: using serial compilation of 6 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
[100%] Built target HardwareSampling
$ cmake --install . --prefix=$HOME/.local
-- Install configuration: "Release"
-- Installing: /home/bzfbnick/.local/lib64/libhardware_sampling.so
-- Installing: /home/bzfbnick/.local/lib64/HardwareSampling.cpython-39-x86_64-linux-gnu.so
-- Set runtime path of "/home/bzfbnick/.local/lib64/HardwareSampling.cpython-39-x86_64-linux-gnu.so" to ""
-- Up-to-date: /home/bzfbnick/.local/include
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling/core.hpp
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling/gpu_amd
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling/gpu_amd/rocm_smi_samples.hpp
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling/gpu_amd/utility.hpp
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling/gpu_amd/hardware_sampler.hpp
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling/event.hpp
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling/gpu_nvidia
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling/gpu_nvidia/nvml_samples.hpp
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling/gpu_nvidia/nvml_device_handle.hpp
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling/gpu_nvidia/utility.hpp
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling/gpu_nvidia/hardware_sampler.hpp
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling/gpu_nvidia/nvml_device_handle_impl.hpp
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling/utility.hpp
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling/hardware_sampler.hpp
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling/gpu_intel
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling/gpu_intel/level_zero_device_handle_impl.hpp
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling/gpu_intel/utility.hpp
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling/gpu_intel/level_zero_samples.hpp
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling/gpu_intel/hardware_sampler.hpp
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling/gpu_intel/level_zero_device_handle.hpp
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling/cpu
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling/cpu/utility.hpp
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling/cpu/hardware_sampler.hpp
-- Up-to-date: /home/bzfbnick/.local/include/hardware_sampling/cpu/cpu_samples.hpp
-- Installing: /home/bzfbnick/.local/share/hardware_sampling/cmake/hardware_samplingTargets.cmake
-- Installing: /home/bzfbnick/.local/share/hardware_sampling/cmake/hardware_samplingTargets-release.cmake
-- Installing: /home/bzfbnick/.local/share/hardware_sampling/cmake/hardware_samplingConfig.cmake
-- Installing: /home/bzfbnick/.local/share/hardware_sampling/cmake/hardware_samplingConfigVersion.cmake
$ CMAKE_INSTALL_PREFIX=$HOME/.local
$ export CMAKE_PREFIX_PATH=${CMAKE_INSTALL_PREFIX}/share/hardware_sampling/cmake:${CMAKE_PREFIX_PATH}
$ export LD_LIBRARY_PATH=${CMAKE_INSTALL_PREFIX}/lib:${LD_LIBRARY_PATH}
$ export CPLUS_INCLUDE_PATH=${CMAKE_INSTALL_PREFIX}/include:${CPLUS_INCLUDE_PATH}
$ export PYTHONPATH=${CMAKE_INSTALL_PREFIX}/lib:${PYTHONPATH}

The CPP CPU example compiles and runs as expected, but the python bindings don't work as expected:

$ python
Python 3.9.19 (main, May  6 2024, 19:43:03)
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import HardwareSampling
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'HardwareSampling'

And just as well the Intel GPU support is disabled. If it helps, I am using OneAPI 2024.2 and Level-Zero v1.17.44, and GCC 13.2.0. The detected python distribution also has pybind11=2.13.6 installed but it is seemingly not found by cmake. Running ccmake . reveals that the following are not found:

level_zero_DIR                   level_zero_DIR-NOTFOUND
pybind11_DIR                     pybind11_DIR-NOTFOUND
subprocess_DIR                subprocess_DIR-NOTFOUND

[breyerml]

Sorry for the late reply, but I didn't got a notification for a new issue.

Regarding the Python issue:

It seems that on your machine, the library files are not installed to ${CMAKE_INSTALL_PREFIX}/lib but to ${CMAKE_INSTALL_PREFIX}/lib64. This directory is automatically determined by GNUInstallDirs's LIBDIR. Unfortunately, the CMake documentation isn't really helpful on understanding when lib or lib64 is used (https://cmake.org/cmake/help/latest/module/GNUInstallDirs.html).

Could you try setting

export LD_LIBRARY_PATH=${CMAKE_INSTALL_PREFIX}/lib64:${LD_LIBRARY_PATH}
export PYTHONPATH=${CMAKE_INSTALL_PREFIX}/lib64:${PYTHONPATH}

I added the lib64 to the README file (820a253).

Regarding the Intel GPU issue:

That's a though one. I've also tried to get Level Zero working on my local machine with an Intel iGPU, but also had major problems. It currently works and is tested on Intel's Tiber Developer Cloud, where everything is preinstalled. I've now added a Findlevel_zero.cmake file (copied from the Developer Cloud) (a0285eb). After adding our cmake directory to the CMAKE_MODULE_PATH and setting export ZES_ENABLE_SYSMAN=1, I was able to use our library on my iGPU. However, the output is rather limited because most samples aren't supported by level zero for my specific iGPU.

I just merged a rather large overhaul onto main, maybe you can check this out too. If this doesn't solve your issues, feel free to contact us again.

[nec4]

No worries - thanks for the response. I will give these suggestions a shot and experiment a little more.

[vancraar]

@nec4 Have you managed to get it working with these fixes?

[nec4]

Hello @vancraar - unfortunately, I have been a bit busy with some other tasks. I am solving already some other issues related to the Intel GPU currently. However, I will try to test this soon and get back to you.

Thanks for your help.