Unitrace build err, shows `CL/cl.h - not found`

[yitingw1]

I am trying to build the unitrace tool on PVC. One PVC succeeded while another failed. I followed https://github.com/intel/pti-gpu/tree/master/tools/unitrace. After the terminal runs cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_WITH_MPI=0 .., it shows Looking for C++ include CL/cl.h - not found.

More details can be found here:

(wyt_tf215) mlp_tf@b4969184c85c:/usnfs/yitingw1/workspace/software/pti-gpu/tools/unitrace/build$ cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_WITH_MPI=0 .. -- The C compiler identification is GNU 12.1.0 -- The CXX compiler identification is GNU 12.1.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /home/mlp_tf/miniconda3/envs/wyt_tf215/bin/x86_64-conda-linux-gnu-cc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /home/mlp_tf/miniconda3/envs/wyt_tf215/bin/x86_64-conda-linux-gnu-c++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Build Type: Release -- Bitness: 64 bits -- Found Xptifw: /home/mlp_tf/intel/oneapi/compiler/2024.0/lib/libxptifw.so
-- OpenCL library is found at /home/mlp_tf/intel/oneapi/compiler/2024.0/lib/libOpenCL.so CMake Warning (dev) at /usnfs/yitingw1/workspace/software/pti-gpu/build_utils/CMakeLists.txt:61 (target_link_libraries): Policy CMP0023 is not set: Plain and keyword target_link_libraries signatures cannot be mixed. Run "cmake --help-policy CMP0023" for policy details. Use the cmake_policy command to set the policy and suppress this warning.

The keyword signature for target_link_libraries has already been used with the target "unitrace_tool". All uses of target_link_libraries with a target should be either all-keyword or all-plain.

The uses of the keyword signature are here:

• CMakeLists.txt:90 (target_link_libraries)

Call Stack (most recent call first): CMakeLists.txt:108 (FindOpenCLLibrary) This warning is for project developers. Use -Wno-dev to suppress it.

-- Looking for C++ include CL/cl.h -- Looking for C++ include CL/cl.h - not found -- Found Python: /home/mlp_tf/miniconda3/envs/wyt_tf215/bin/python3.10 (found version "3.10.13") found components: Interpreter -- OpenCL headers are not found, will be downloaded automatically -- /home/mlp_tf/miniconda3/envs/wyt_tf215/bin/python3.10 /usnfs/yitingw1/workspace/software/pti-gpu/build_utils/get_cl_headers.py -- /usnfs/yitingw1/workspace/software/pti-gpu/tools/unitrace/build /usnfs/yitingw1/workspace/software/pti-gpu/tools/unitrace/build CMake Error at /usnfs/yitingw1/workspace/software/pti-gpu/build_utils/CMakeLists.txt:688 (message): Level Zero loader is not found. You may need to install oneAPI Level Zero loader to fix this issue. Call Stack (most recent call first): CMakeLists.txt:113 (FindL0Library)

-- Configuring incomplete, errors occurred! See also "/usnfs/yitingw1/workspace/software/pti-gpu/tools/unitrace/build/CMakeFiles/CMakeOutput.log". See also "/usnfs/yitingw1/workspace/software/pti-gpu/tools/unitrace/build/CMakeFiles/CMakeError.log".

I have seen 'CMakeError.log' which shows fatal error: CL/cl.h: No such file or directory. I have tried sudo apt-get install opencl-headers which is useless.

Then I looked into the unitrace/CMakeLists.txt and found the error occured in Line109 FindOpenCLHeaders(unitrace_tool). It was in macro(FindOpenCLHeaders TARGET) in build_utils/CMakeLists.txt. The error occurs in line77 when CHECK_INCLUDE_FILE_CXX(CL/cl.h OpenCL_INCLUDE_DIRS) fails to find CL/cl.h. Then it runs if-statement in line 89. It is supposed to download cl_headers to ${CMAKE_BINARY_DIR} through add_custom_command in line100 but it fails to do that. I have checked ${CMAKE_BINARY_DIR}, there is no OpenCL-Headers Dir. And even I downloaded OpenCL-Headers Dir and got CL Dir through python get_ocl_headers.py <include_path> <build_path>, cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_WITH_MPI=0 .. was still failed with the same error.

[Sarbojit2019]

@yitingw1, Looks like actual problem is below error not "CL/cl.h file is missing" which is analyzed above. I have captured below log from CMake output you have pasted above.

CMake Error at /usnfs/yitingw1/workspace/software/pti-gpu/build_utils/CMakeLists.txt:688 (message): Level Zero loader is not found. You may need to install oneAPI Level Zero loader to fix this issue.

CL/cl.h file missing gets resolved during 'make' hence you should ignore it. Make sure your PVC system has oneAPI tools and L0 installed correctly and load oneAPI environment before unitrace build.

Let me know if you still see the issue.

[yitingw1]

@Sarbojit2019 I'm sure the PVC system has oneAPI tools and L0 installed correctly and load oneAPI environment before unitrace build.

source /opt/intel/oneapi/setvars.sh

:: initializing oneAPI environment ...
bash: BASH_VERSION = 5.1.16(1)-release
args: Using "$@" for setvars.sh arguments: 
:: advisor -- latest
:: ccl -- latest
:: compiler -- latest
:: dal -- latest
:: debugger -- latest
:: dev-utilities -- latest
:: dnnl -- latest
:: dpcpp-ct -- latest
:: dpl -- latest
:: ipp -- latest
:: ippcp -- latest
:: mkl -- latest
:: mpi -- latest
:: tbb -- latest
:: vtune -- latest
:: oneAPI environment initialized ::

dpkg -l | grep -i level-zero

ii  intel-level-zero-gpu                            1.3.27642.40-803~22.04                                          amd64        Intel(R) Graphics Compute Runtime for oneAPI Level Zero.
ii  intel-level-zero-gpu-dev                        1.3.27642.40-803~22.04                                          amd64        Intel(R) Graphics Compute Runtime for oneAPI Level Zero
ii  intel-level-zero-gpu-raytracing                 1.0.0~u22.04                                                    amd64        Level Zero Ray Tracing Support library
ii  level-zero                                      1.14.0-744~22.04                                                amd64        Intel(R) Graphics Compute Runtime for oneAPI Level Zero.
ii  level-zero-dev                                  1.14.0-744~22.04                                                amd64        Intel(R) Graphics Compute Runtime for oneAPI Level Zero.

I'm afraid that you may not fully understand my question. I know CL/cl.h file missing gets resolved during 'make'. What I have seen is that the makefile may not deal with it properly.

I looked into the unitrace/CMakeLists.txt and found the error occured in Line109 FindOpenCLHeaders(unitrace_tool). It was in macro(FindOpenCLHeaders TARGET) in build_utils/CMakeLists.txt. The error occurs in line77 when CHECK_INCLUDE_FILE_CXX(CL/cl.h OpenCL_INCLUDE_DIRS) fails to find CL/cl.h. Then it runs if-statement in line 89. It is supposed to download cl_headers to ${CMAKE_BINARY_DIR} through add_custom_command in line100 but it fails to do that. I have checked ${CMAKE_BINARY_DIR}, there is no OpenCL-Headers Dir. And even I downloaded OpenCL-Headers Dir and got CL Dir through python get_ocl_headers.py , cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_WITH_MPI=0 .. was still failed with the same error.

[Sarbojit2019]

@yitingw1, In your first response the log shared has Level-zero error but you are referring OpenCL error which is very confusing. Please share the build log (cmake and make) along with command you have used to run them.

Unless I see the error log it is hard to guess why did it fail even after supplying header files which were needed (as you mentioned in your comment).

[yitingw1]

@Sarbojit2019 Thanks for classifying the difference between successful logs and failed logs. They use different GNU and C/CXX compilers.

• The successful log uses GNU 11.4.0 and C(CXX) compilers in /usr/bin/cc(c++)
• The failed one uses GNU12.1.0 and C(CXX) compilers in ~/miniconda3/envs/wyt_tf215/bin/x86_64-conda-linux-gnu-cc(c++), which is in conda env.

After using GNU 11.4.0 and C(CXX) compilers in /usr/bin/cc(c++), unitrace can build successfully. Both CL/cl.h and Level Zero loader can be found now.

[Sarbojit2019]

@yitingw1, Glad to hear that problem is resolved. Can you close the ticket please?