'lldELF_PATH-NOTFOUND', needed by 'IGC/Release/elf_packager', missing and no known rule to make it

[eero-t]

On Ubuntu 20.04 with vc-intrinsics master version and rest of deps from Ubuntu (which has built earlier fine e.g. in May), latest release errors after configuring to:

Step 30/61 : RUN git clone --branch ${TAG_IGC} --depth 1 https://github.com/intel/intel-graphics-compiler.git  &&     cd intel-graphics-compiler  &&     mkdir build  &&  cd build  &&      cmake -LH -Wno-dev -G Ninja       -DCMAKE_INSTALL_PREFIX=${INSTALL_DIR}       -DCMAKE_BUILD_TYPE=Release       -DIGC_OPTION__LLVM_PREFERRED_VERSION=10       -DINSTALL_SPIRVDLL=0
...
ninja: error: 'lldELF_PATH-NOTFOUND', needed by 'IGC/Release/elf_packager', missing and no known rule to make it

Here's the whole output before that error:

 ---> Running in 5fd7e9aa04dc
Cloning into 'intel-graphics-compiler'...

Note: switching to '4812c40efcae76bc7fd3dec353212b6c4646c1eb'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

-- The C compiler identification is GNU 10.3.0
-- The CXX compiler identification is GNU 10.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Warning at IGC/CMakeLists.txt:158 (message):
  IGC_OPTION__ARCHITECTURE_TARGET: Target architecture is not specified.
  Trying to deduce it from context.

-- ================================ IGC Project ================================
-- Build type:  Release (single-configuration generator)
-- Build tools: OFF
-- LLVM dir:    
-- 
-- Output directory:
--    "/home/nobody/source/intel-graphics-compiler/build/IGC/Release"
-- Install root directory:
--    "/usr/local"
-- Architecture:
--  - target: Linux64    (detected: Linux64)
--  - host:   Linux64    (detected: Linux64)
-- 
-- Cross-compilation needed: NO
-- Cross-compilation set:    FALSE
-- 
-- Advanced:
--  - Link BiF resources:              ON
--  - Building Windows Universal:      
-- =============================================================================

-- [Clang] No mode was specified, searching for Clang
-- [lld] No mode was specified, searching for lld
-- [LLVM] No LLVM mode was selected explicitly
-- [LLVM] Search will be performed for LLVM sources first, then for prebuilds (including system LLVM)
-- [IGC] IGC will take prebuilt LLVM
-- [IGC] Searching for prebuilt LLVM in /home/nobody/source/intel-graphics-compiler/IGC/../../prebuild-llvm/Release and system directories
-- [IGC] Found LLVM: /usr/lib/llvm-10/cmake
-- Linker detection: GNU ld
-- [IGC] : opencl-clang will be taken from system
-- [IGC] Found clang-10 executable: /usr/bin/clang-10
-- [CMFE] CM Frontend headers are taken from: /home/nobody/source/intel-graphics-compiler/IGC/CMFE/AdaptorCM
-- [CMFE] Using CMFE library name: libclangFEWrapper.so
-- +++ Source/IGC/VectorCompiler +++
-- [VC] Build proj: VectorCompiler
-- [VC] Using vc-intrinsics source from: /home/nobody/source/intel-graphics-compiler/IGC/VectorCompiler/../../../vc-intrinsics/GenXIntrinsics
 >>/home/nobody/source/intel-graphics-compiler/build/IGC/VectorCompiler/lib/GenXCodeGen/Utils/../<<  ->  /home/nobody/source/intel-graphics-compiler/build/IGC/VectorCompiler/lib/GenXCodeGen/Utils
 COMMAND ->  /usr/bin/python3 /home/nobody/source/intel-graphics-compiler/IGC/VectorCompiler/lib/GenXCodeGen/Utils/cisa_gen_intrinsics.py /home/nobody/source/intel-graphics-compiler/IGC/VectorCompiler/lib/GenXCodeGen/Utils/cisa_gen_intrinsics.json /home/nobody/source/intel-graphics-compiler/build/IGC/VectorCompiler/lib/GenXCodeGen/Utils/../ <-
 /home/nobody/source/intel-graphics-compiler/IGC/VectorCompiler/lib/GenXCodeGen/Utils
vc_embed_bif - VCBiFPrintfOCL32 has extra dependencies: /home/nobody/source/intel-graphics-compiler/IGC/VectorCompiler/lib/BiF/printf_not_cm_common.h
vc_embed_bif - VCBiFPrintfOCL64 has extra dependencies: /home/nobody/source/intel-graphics-compiler/IGC/VectorCompiler/lib/BiF/printf_not_cm_common.h
vc_embed_bif - VCBiFPrintfZE32 has extra dependencies: /home/nobody/source/intel-graphics-compiler/IGC/VectorCompiler/lib/BiF/printf_not_cm_common.h
vc_embed_bif - VCBiFPrintfZE64 has extra dependencies: /home/nobody/source/intel-graphics-compiler/IGC/VectorCompiler/lib/BiF/printf_not_cm_common.h
vc_embed_bif - VCEmulation64 has extra dependencies: /home/nobody/source/intel-graphics-compiler/build/IGC/VectorCompiler/lib/BiF/Utils/emulation_divrem_boilerplate.h
Use InstCombiner from LLVM
-- Configuring Intel Gen Assembler (IGA) Component
--  - GED_BRANCH:           GED_external
--  - CMAKE_CXX_COMPILER:   /usr/bin/c++
--  - IGA_VERSION_SUFFIX:    -4812c40
-- Found BISON: /usr/bin/bison (found version "3.7") 
-- Found FLEX: /usr/bin/flex (found version "2.6.4") 
[check-igc] LIT tests disabled. Missing igc_opt target.
-- Configuring done
-- Generating done
-- Build files have been written to: /home/nobody/source/intel-graphics-compiler/build
-- Cache values
// Path to a library.
CCLANG_FROM_SYSTEM:FILEPATH=/usr/lib/libopencl-clang.so

// Path to a program.
CLANG_GE7:FILEPATH=/usr/bin/clang-10

// Choose the type of build, options are: None Debug Release RelWithDebInfo MinSizeRel ...
CMAKE_BUILD_TYPE:STRING=Release

// Install path prefix, prepended onto install directories.
CMAKE_INSTALL_PREFIX:PATH=/usr/local

// Path to a file.
DEFAULT_IGC_LLVM_SOURCES_DIR:PATH=DEFAULT_IGC_LLVM_SOURCES_DIR-NOTFOUND

// Path to a file.
FOUND_VCS:FILEPATH=/usr/lib/llvm-10/lib/cmake/llvm/VersionFromVCS.cmake

// IGC available build configurations.
IGC_CMAKE_CONFIGURATION_TYPES:STRING=Debug;Release

// Architecture of host used during compilation. Only used when cross-compiling.
IGC_OPTION__ARCHITECTURE_HOST:STRING=Linux64

// Target architecture for compilation.
IGC_OPTION__ARCHITECTURE_TARGET:STRING=Linux64

// Build project igc_opt.
IGC_OPTION__BUILD_IGC_OPT:BOOL=ON

// Clang mode for IGC (can be Source, Prebuilds or empty)
IGC_OPTION__CLANG_MODE:STRING=

// Enable lit testing for IGC compiler. May require additional tools like llvm lit and opt
IGC_OPTION__ENABLE_LIT_TESTS:BOOL=OFF

// Only building igcc part of IST (IGCStandalone).
IGC_OPTION__IST_IGCC_ONLY:BOOL=ON

// [Experimental] Enable link against Khronos SPIRV-LLVM-Translator, instead of legacy versions
IGC_OPTION__LINK_KHRONOS_SPIRV_TRANSLATOR:BOOL=OFF

// LLD mode for IGC (can be Source, Prebuilds or empty)
IGC_OPTION__LLD_MODE:STRING=

// LLVM mode for IGC (can be Source, Prebuilds or empty)
IGC_OPTION__LLVM_MODE:STRING=

// Preferred version of LLVM to use
IGC_OPTION__LLVM_PREFERRED_VERSION:STRING=10

// Output directory path where the final libraries will be stored.
IGC_OPTION__OUTPUT_DIR:PATH=/home/nobody/source/intel-graphics-compiler/build/IGC/Release

// [Experimental] Enable usage of Khronos SPIRV-LLVM-Translator in Scalar Compiler
IGC_OPTION__USE_KHRONOS_SPIRV_TRANSLATOR_IN_SC:BOOL=OFF

// [Experimental] Enable usage of Khronos SPIRV-LLVM-Translator in Vector Compiler
IGC_OPTION__USE_KHRONOS_SPIRV_TRANSLATOR_IN_VC:BOOL=OFF

// disable BiF generation for VC (disables some BE features)
IGC_OPTION__VC_DISABLE_BIF:BOOL=OFF

// install GENX_IR or not
INSTALL_GENX_IR:BOOL=ON

// link with /MT or /MD
LINK_AS_STATIC_LIB:BOOL=ON

// Path to a file.
LLD_INCLUDE_DIR:PATH=LLD_INCLUDE_DIR-NOTFOUND

// The directory containing a CMake configuration file for LLVM.
LLVM_DIR:PATH=/usr/lib/llvm-10/cmake

// Path to a program.
PYTHON_EXECUTABLE:FILEPATH=/usr/bin/python3

// Path to a library.
SYSTEM_COMMON_CLANG:FILEPATH=/usr/lib/libopencl-clang.so

// Path to a library.
lldCommon_PATH:FILEPATH=lldCommon_PATH-NOTFOUND

// Path to a library.
lldELF_PATH:FILEPATH=lldELF_PATH-NOTFOUND

ninja: error: 'lldELF_PATH-NOTFOUND', needed by 'IGC/Release/elf_packager', missing and no known rule to make it

[paigeale]

@eero-t thank you for opening this issue. We are debugging internally and will get back to you shortly.

[lwesiers]

Hello there,

Probably you are missing those packages: https://packages.ubuntu.com/focal/liblld-10 https://packages.ubuntu.com/focal/liblld-10-dev

Could you install them and give another try?

[eero-t]

Yes, configuration phase passes with that being present => Please add that info also to build instructions!

However, after that, there's another regression.

Build fails to linking error, I guess because pthreads is not listed in CMake linker options:

[795/796] Linking CXX shared library IGC/Release/libigc.so.1.0.1
FAILED: IGC/Release/libigc.so.1.0.1 
: && /usr/bin/c++ -fPIC -fno-exceptions -fdata-sections -ffunction-sections -O2 -pipe -fmessage-length=0 -march=corei7 -mstackrealign -fms-extensions -Werror -Wno-unused-parameter -Wno-missing-field-initializers -Wwrite-strings -Wno-long-long -Wswitch -Wno-sign-compare -Wno-unused-result -Wno-enum-compare -Wno-type-limits -Wno-ignored-qualifiers -Wno-shadow -Wformat -Wformat-security -Wno-extra -Wno-write-strings -finline -fno-strict-aliasing -msse -msse2 -msse3 -mssse3 -msse4 -msse4.1 -msse4.2 -Wno-unknown-pragmas -fPIC -D_FORTIFY_SOURCE=2 -fstack-protector -finline-functions -funswitch-loops -Wno-maybe-uninitialized -lrt -fno-rtti -fvisibility=hidden -fvisibility-inlines-hidden -DNDEBUG -g  -Wl,--no-undefined -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,--gc-sections -shared -Wl,-soname,libigc.so.1 -o IGC/Release/libigc.so.1.0.1 IGC/CMakeFiles/igc_dll.dir/common/CompilerStatsUtils.cpp.o IGC/CMakeFiles/igc_dll.dir/common/igc_regkeys.cpp.o IGC/CMakeFiles/igc_dll.dir/common/IGCConstantFolder.cpp.o IGC/CMakeFiles/igc_dll.dir/common/LLVMUtils.cpp.o IGC/CMakeFiles/igc_dll.dir/common/ShaderOverride.cpp.o IGC/CMakeFiles/igc_dll.dir/common/Stats.cpp.o IGC/CMakeFiles/igc_dll.dir/common/SysUtils.cpp.o IGC/CMakeFiles/igc_dll.dir/common/debug/Debug.cpp.o IGC/CMakeFiles/igc_dll.dir/common/debug/Dump.cpp.o IGC/CMakeFiles/igc_dll.dir/common/debug/TeeOutputStream.cpp.o IGC/CMakeFiles/igc_dll.dir/common/SystemThread.cpp.o IGC/CMakeFiles/igc_dll.dir/common/MDFrameWork.cpp.o IGC/CMakeFiles/igc_dll.dir/common/FunctionUpgrader.cpp.o IGC/CMakeFiles/igc_dll.dir/common/IntrinsicAnnotator.cpp.o IGC/CMakeFiles/igc_dll.dir/common/allocator.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/CLElfLib/ElfReader.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/CLElfLib/ElfWriter.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorCommon/AddImplicitArgs.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorCommon/customApi.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/dllInterfaceCompute.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorCommon/ImplicitArgs.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorCommon/ProcessFuncAttributes.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorCommon/TypesLegalizationPass.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorCommon/LegalizeFunctionSignatures.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/OCL/LoadBuffer.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/OCL/Patch/patch_parser.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/OCL/Platform/cmd_media_caps_g8.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/OCL/Platform/cmd_parser_g8.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/OCL/sp/spp_g8.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/OCL/sp/sp_g8.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/OCL/sp/sp_debug.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/OCL/util/BinaryStream.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/UnifyIROCL.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/MoveStaticAllocas.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/OCL/sp/zebin_builder.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/SPIRV/libSPIRV/SPIRVBasicBlock.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/SPIRV/libSPIRV/SPIRVDebug.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/SPIRV/libSPIRV/SPIRVDecorate.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/SPIRV/libSPIRV/SPIRVEntry.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/SPIRV/libSPIRV/SPIRVFunction.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/SPIRV/libSPIRV/SPIRVInstruction.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/SPIRV/libSPIRV/SPIRVModule.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/SPIRV/libSPIRV/SPIRVStream.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/SPIRV/libSPIRV/SPIRVType.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/SPIRV/libSPIRV/SPIRVValue.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/SPIRV/SPIRVReader.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/SPIRV/SPIRVUtil.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/ocl_igc_interface/impl/igc_features_and_workarounds_impl.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/ocl_igc_interface/impl/igc_ocl_device_ctx_impl.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/ocl_igc_interface/impl/igc_ocl_translation_ctx_impl.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/ocl_igc_interface/impl/ocl_gen_binary_impl.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/ocl_igc_interface/impl/ocl_translation_output_impl.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/ocl_igc_interface/impl/gt_system_info_impl.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/ocl_igc_interface/impl/platform_impl.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/cif/cif/builtins/memory/buffer/impl/buffer_impl.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/cif/cif/builtins/builtins_registry.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/cif/cif/export/cif_main.cpp.o IGC/CMakeFiles/igc_dll.dir/AdaptorOCL/cif/cif/export/registry.cpp.o  -Wl,-rpath,/usr/lib/llvm-10/lib:  IGC/Release/libzebinlib.a  IGC/Release/libCompiler.a  IGC/Release/libGenISAIntrinsics.a  IGC/visa/igfxcmjit64.a  IGC/Release/libGenXDebugInfo.a  -Wl,--whole-archive  IGC/Release/libBiFLibOcl.a  -Wl,--no-whole-archive  IGC/Release/libVCIGCDeps.a  -Wl,--start-group  /usr/lib/llvm-10/lib/liblldELF.a  -Wl,--end-group  -ldl  IGC/Release/libVCDriver.a  -ldl  IGC/Release/libIGCOptions.a  IGC/Release/libVCCodeGen.a  IGC/visa/igfxcmjit64.a  IGC/Release/libiga_enc64.a  IGC/Release/libiga_s64.a  IGC/Release/libGenXDebugInfo.a  IGC/Release/libVCTransforms.a  IGC/Release/libVCTargetInfo.a  IGC/Release/libVCGeneralUtils.a  IGC/Release/libVCGenXUtils.a  IGC/Release/libLLVMGenXIntrinsics.a  IGC/Release/libVCSupport.a  IGC/Release/libVCEmbeddedBiF.a  /usr/lib/llvm-10/lib/liblldCommon.a  /usr/lib/llvm-10/lib/libLLVM-10.so.1 && :
/usr/bin/ld: /usr/lib/llvm-10/lib/liblldCommon.a(Filesystem.cpp.o): in function `lld::unlinkAsync(llvm::StringRef)':
(.text._ZN3lld11unlinkAsyncEN4llvm9StringRefE+0x14e): undefined reference to `pthread_create'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

[eero-t]

Updating to "igc-1.0.7862" (released yesterday) did not solve the link issue.

[lwesiers]

Please try add pthread to this function: https://github.com/intel/intel-graphics-compiler/blob/master/IGC/CMakeLists.txt#L1700

else()
target_link_libraries("${IGC_BUILD__PROJ${_libBuildSuffix}}" PRIVATE
    pthread // here add
    "${_targetLinkLineCommon}"
    "${CMAKE_DL_LIBS}"
  )
endif()

and let me know if this helps.

[eero-t]

Yes, after that IGC does build successfully. Thanks!

This can be closed after that fix is in, and build doc has been updated in regards to IGC now needing lld-XX.

I can verify the fix when the next release is done.

[eero-t]

Regarding this line in the doc update: sudo apt-get install llvm-10 llvm-10-dev clang-10 liblld-10 liblld-10-dev

'-dev' packages pull in the non-dev version of the given package automatically, so specifying those is redundant. Or is the intent to set them as manually-installed, so that they do not automatically go away when corresponding -dev packages are removed (and user runs "apt autoremove")?

[lwesiers]

they do not automatically go away when corresponding -dev packages are removed (and user runs "apt autoremove")?

yes