search

intel / intel-graphics-compiler

Other
597 stars 155 forks source link

SPIRVDLL: request to use system-installed SPIRV-LLVM-Translator #148

Closed tgurr closed 2 years ago

tgurr commented 4 years ago

I'm trying to update our distribution packages to a recent compute-runtime version and now run into build errors for intel-graphics-compiler. There seem to be two new dependencies:

-- [VC] Cannot find SPIRVDLL sources in /var/tmp/paludis/build/dev-util-intel-graphics-compiler-1.0.4756/work/intel-graphics-compiler-igc-1.0.4756/IGC/VectorCompiler/../../../llvm-project/llvm/projects/llvm-spirv
-- Configuring incomplete, errors occurred!

Is there any way to use an already installed SPIRV-LLVM-Translator like we do for everything else, LLVM, opencl-clang and stuff?

Same question for vc-intrinsics.

kv-sc commented 4 years ago

Can you please provide cmake line how are you trying to configure and build?

tgurr commented 4 years ago

Sure, it's:

cmake -DCMAKE_COLOR_MAKEFILE:BOOL=TRUE -DCMAKE_VERBOSE_MAKEFILE:BOOL=TRUE -DCMAKE_BUILD_TYPE:STRING=None -DCMAKE_C_FLAGS:STRING=-march=native -O2 -pipe -DCMAKE_CXX_FLAGS:STRING=-march=native -O2 -pipe -DCMAKE_AR:PATH=x86_64-pc-linux-gnu-ar -DCMAKE_RANLIB:PATH=x86_64-pc-linux-gnu-ranlib -DCMAKE_NM:PATH=x86_64-pc-linux-gnu-nm -DCMAKE_C_COMPILER:PATH=x86_64-pc-linux-gnu-cc -DCMAKE_CXX_COMPILER:PATH=x86_64-pc-linux-gnu-c++ -DCMAKE_INSTALL_PREFIX:PATH=/usr/x86_64-pc-linux-gnu -DCMAKE_FIND_ROOT_PATH=/usr/x86_64-pc-linux-gnu -DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM:STRING=NEVER -DCMAKE_SYSTEM_PREFIX_PATH:PATH=/usr/x86_64-pc-linux-gnu -DCMAKE_INSTALL_LIBDIR:STRING=lib -DCMAKE_INSTALL_DATAROOTDIR:PATH=/usr/share/ -DCCLANG_FROM_SYSTEM:BOOL=TRUE -DIGC_PREFERRED_LLVM_VERSION:STRING=10 -DLLVM_DIR:PATH=/usr/x86_64-pc-linux-gnu/lib/llvm/10/lib/cmake/llvm /var/tmp/paludis/build/dev-util-intel-graphics-compiler-1.0.4756/work/intel-graphics-compiler-igc-1.0.4756

Complete build log: intel-graphics-compiler-1.0.4756_build.log

kv-sc commented 4 years ago

I consulted with our infra guys and now this looks not possible (at least in short term)

We are using spirv as shared object because IGC already have copy of legacy SPIRV sources and we need vanilla translator for Vector backend.

Since vanilla translator needs patch to be able to be build as shared object, we need sources to apply this patch

This will be the case at least as long as scalar path in IGC will not be upgraded to vanilla translator as well and then we will be able to reuse the same one in our VC

eero-t commented 3 years ago

IGC (and compute-runtime after that) build fine for me on Ubuntu 20.04 with system LLVM & SPRIV, when using these options: -DIGC_PREFERRED_LLVM_VERSION=10 -DIGC_OPTION__FORCE_SYSTEM_LLVM=1 -DINSTALL_SPIRVDLL=0

"vc-intrinsics" sources seem to be needed, but that's a small thing unlike LLVM/SPIRV stuff.

PS. I install these packages to build the whole stack: gcc g++ flex bison autoconf automake libtool build-essential pkg-config make meson ninja-build valgrind clang-10 libopencl-clang-dev opencl-headers llvm-10-dev llvm-spirv libllvmspirvlib-dev libz-dev libpciaccess-dev.

ArchangeGabriel commented 3 years ago

@eero-t Thanks for the hint about INSTALL_SPIRVDLL. Looking at the source code, LLVM SPIRV Translator sources are only required for debug, which is enabled by default but disabled by this flag.

For vc-intrinsics sources being required, there is already a ticket at #142.

thiagomacieira commented 3 years ago

This should be made automatic. Allowing failing builds because LLVMSPIRV is already installed is a bug.

eero-t commented 3 years ago

IGC (and compute-runtime after that) build fine for me on Ubuntu 20.04 with system LLVM & SPRIV, when using these options: -DIGC_PREFERRED_LLVM_VERSION=10 -DIGC_OPTION__FORCE_SYSTEM_LLVM=1 -DINSTALL_SPIRVDLL=0

This has regressed. IGC CMake option names seem to have changed in latest release(s), so above doesn't work any more.

However, using the new option names doesn't help either (I've tried all the variants of the SPIRV option values).

Build fails now always to IGC complaining about not finding SPIRVDLL sources (this is with Ubuntu 21.04 which has llvm/clang v11 instead of v10):

Step 30/61 : RUN git clone --branch ${TAG_IGC} --depth 1 https://github.com/intel/intel-graphics-compiler.git  && \
    cd intel-graphics-compiler  &&  mkdir build  &&  cd build  && \
    cmake -LH -Wno-dev -G Ninja \
      -DCMAKE_INSTALL_PREFIX=${INSTALL_DIR} \
      -DCMAKE_BUILD_TYPE=Release \
      -DIGC_OPTION__LLVM_PREFERRED_VERSION=11 \
      -DIGC_OPTION__LLVM_MODE=Prebuilds \
      -DIGC_OPTION__CLANG_MODE=Prebuilds \
      -DIGC_OPTION__USE_KHRONOS_SPIRV_TRANSLATOR=ON \
      -DIGC_OPTION__SPIRV_TRANSLATOR_MODE=Prebuilds \
      ../  &&     ninja  &&  ninja install
 ---> Running in 42ee472a9090

Cloning into 'intel-graphics-compiler'...
...
-- The C compiler identification is GNU 10.3.0
-- The CXX compiler identification is GNU 10.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done

-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Warning at IGC/CMakeLists.txt:158 (message):
  IGC_OPTION__ARCHITECTURE_TARGET: Target architecture is not specified.
  Trying to deduce it from context.

-- ================================ IGC Project ================================
-- Build type:  Release (single-configuration generator)
-- Build tools: OFF
-- LLVM dir:    
-- 
-- Output directory:
--    "/home/nobody/source/intel-graphics-compiler/build/IGC/Release"
-- Install root directory:
--    "/usr/local"
-- Architecture:
--  - target: Linux64    (detected: Linux64)
--  - host:   Linux64    (detected: Linux64)
-- 
-- Cross-compilation needed: NO
-- Cross-compilation set:    FALSE
-- 
-- Advanced:
--  - Link BiF resources:              ON
--  - Building Windows Universal:      
-- =============================================================================
-- [IGC] IGC will take prebuilt LLVM
-- [IGC] Searching for prebuilt LLVM in /home/nobody/source/intel-graphics-compiler/IGC/../../prebuild-llvm/Release and system directories
-- [IGC] Found LLVM: /usr/lib/llvm-11/cmake
-- Linker detection: GNU ld
-- [IGC] Trying to find prebuilt SPIRV library
-- Found SPIRVLLVMTranslator: /usr/lib/libLLVMSPIRVLib.so (found suitable version "11.0.0.0", minimum required is "11")  
-- [IGC] : opencl-clang will be taken from system
-- [IGC] Found clang-11 executable: /usr/bin/clang-11
-- [CMFE] CM Frontend headers are taken from: /home/nobody/source/intel-graphics-compiler/IGC/CMFE/AdaptorCM
-- [CMFE] Using CMFE library name: libclangFEWrapper.so
-- +++ Source/IGC/VectorCompiler +++
-- [VC] Build proj: VectorCompiler
-- [VC] Using vc-intrinsics source from: /home/nobody/source/intel-graphics-compiler/IGC/VectorCompiler/../../../vc-intrinsics/GenXIntrinsics
-- [VC] SPIRVDLL_SRC = 
-- [VC] SPIRV_SRC = 
-- [VC] Cannot find SPIRVDLL sources in /home/nobody/source/intel-graphics-compiler/IGC/VectorCompiler/../../../llvm-project/llvm/projects/llvm-spirv
CMake Error at IGC/VectorCompiler/cmake/spirv.cmake:152 (message):
  [VC] Cannot find SPIRVDLL sources in
  /home/nobody/source/intel-graphics-compiler/IGC/VectorCompiler/../../../llvm-spirv
Call Stack (most recent call first):
  IGC/VectorCompiler/CMakeLists.txt:78 (include)

-- Configuring incomplete, errors occurred!
See also "/home/nobody/source/intel-graphics-compiler/build/CMakeFiles/CMakeOutput.log".
-- Cache values
// Path to a library.
CCLANG_FROM_SYSTEM:FILEPATH=/usr/lib/libopencl-clang.so

// Path to a program.
CLANG_GE7:FILEPATH=/usr/bin/clang-11

// Choose the type of build, options are: None Debug Release RelWithDebInfo MinSizeRel ...
CMAKE_BUILD_TYPE:STRING=Release

// Install path prefix, prepended onto install directories.
CMAKE_INSTALL_PREFIX:PATH=/usr/local

// Path to a file.
DEFAULT_IGC_LLVM_SOURCES_DIR:PATH=DEFAULT_IGC_LLVM_SOURCES_DIR-NOTFOUND

// Path to a file.
DEFAULT_SPIRV_TRANSLATOR_SOURCE_DIR:PATH=DEFAULT_SPIRV_TRANSLATOR_SOURCE_DIR-NOTFOUND

// Path to a file.
FOUND_VCS:FILEPATH=/usr/lib/llvm-11/lib/cmake/llvm/VersionFromVCS.cmake

// IGC available build configurations.
IGC_CMAKE_CONFIGURATION_TYPES:STRING=Debug;Release

// Architecture of host used during compilation. Only used when cross-compiling.
IGC_OPTION__ARCHITECTURE_HOST:STRING=Linux64

// Target architecture for compilation.
IGC_OPTION__ARCHITECTURE_TARGET:STRING=Linux64

// Build project igc_opt.
IGC_OPTION__BUILD_IGC_OPT:BOOL=ON

// Clang mode for IGC (can be Source, Prebuilds or empty)
IGC_OPTION__CLANG_MODE:STRING=Prebuilds

// Enable lit testing for IGC compiler. May require additional tools like llvm lit and opt
IGC_OPTION__ENABLE_LIT_TESTS:BOOL=OFF

// Only building igcc part of IST (IGCStandalone).
IGC_OPTION__IST_IGCC_ONLY:BOOL=ON

// LLVM mode for IGC (can be Source, Prebuilds or empty)
IGC_OPTION__LLVM_MODE:STRING=Prebuilds

// Preferred version of LLVM to use
IGC_OPTION__LLVM_PREFERRED_VERSION:STRING=11

// Output directory path where the final libraries will be stored.
IGC_OPTION__OUTPUT_DIR:PATH=/home/nobody/source/intel-graphics-compiler/build/IGC/Release

// SPIRV mode for IGC (can be Source, Prebuilds or empty)
IGC_OPTION__SPIRV_TRANSLATOR_MODE:STRING=Prebuilds

// [Experimental] Enable link against Khronos SPIRV-LLVM-Translator, instead of legacy versions
IGC_OPTION__USE_KHRONOS_SPIRV_TRANSLATOR:BOOL=ON

// The directory containing a CMake configuration file for LLVM.
LLVM_DIR:PATH=/usr/lib/llvm-11/cmake

// Paths to LLVM prebuild (multiple paths can be specified separated by ;
LLVM_ROOT:PATH=/home/nobody/source/intel-graphics-compiler/IGC/../../prebuild-llvm/Release

// Path to a program.
PYTHON_EXECUTABLE:FILEPATH=/usr/bin/python3

// Path to a library.
SYSTEM_COMMON_CLANG:FILEPATH=/usr/lib/libopencl-clang.so
eero-t commented 3 years ago

Sorry, building still works with latest IGC release (igc-1.0.7423) when using: -DIGC_OPTION__LLVM_PREFERRED_VERSION=10 -DINSTALL_SPIRVDLL=0

Apparently INSTALL_SPIRVDLL option wasn't deprecated, for some reason it just does not show up in in CMake -LH output...

(I also downgraded back to Ubuntu 20.10 / LLVM v10, as latest IGC release build still seems to fail with LLVM v11.)

PS. regarding other possible system dependencies... According to info I got internally earlier, "opencl-clang" releases v10 & v11 (which are from Oct 2020) have not been new enough for IGC for long time. Having newer "opencl-clang" version (at least Dec 8, 2020) is critical for OpenCL v3.0 support.

(And IGC project anyway recommends using the exact commit IDs listed in release notes for all IGC deps, as it is fully tested to work only against those versions.)

ArchangeGabriel commented 3 years ago

They are v12 releases in the stack now, for LLVM 12. That’s what we use, with patches to fix support in IGC: https://github.com/archlinux/svntogit-community/blob/packages/intel-graphics-compiler/trunk/PKGBUILD

eero-t commented 3 years ago

To be more precise, while "llvm-spirv" & "opencl-clang" in 21.04 are LLVM v11 based, Ubuntu 21.04 LLVM default is v12. And in Ubuntu 20.10, version numbers for all are -1.

While I failed to build last IGC release against LLVM v11 on Ubuntu 21.04, IGC package already in 21.04 does use LLVM v11. However, that and Debian package it's derived from, are from last October i.e. really old and disable VC (I did not): https://salsa.debian.org/opencl-team/intel-graphics-compiler/-/blob/master/debian/rules

According to above Arch package link, Arch doesn't disable VC with LLVM v12. Debian package disabled VC, when it upgraded to 1.0.5064: https://metadata.ftp-master.debian.org/changelogs//main/i/intel-graphics-compiler/intel-graphics-compiler_1.0.5353.1-2_changelog

PS. No Ubuntu version has LLVM v12 based "llvm-spirv" or "opencl-clang" packages yet, they are only in Debian experimental so far.

aus-intel commented 3 years ago

After https://github.com/intel/intel-graphics-compiler/commit/7309126ed72fb6136ff0ff1ee522121e97fb07a2 it should be possible to build IGC without SPIRVDLL using original Khronos SPIRV translator. This is controlled by main options that switch VC to use translator without modifications: -DIGC_OPTION__LINK_KHRONOS_SPIRV_TRANSLATOR=ON and -DIGC_OPTION__USE_KHRONOS_SPIRV_TRANSLATOR_IN_VC=ON. Mode for SPIRV is controlled by -DIGC_OPTION__SPIRV_TRANSLATOR_MODE that can be set to either Source or Prebuilds. In Prebuilds mode IGC will try to search for prebuilt SPIRV translator library. Exact location of prebuilds can additionally be controlled by -DSPIRVLLVMTranslator_ROOT=<path to spirv install dir> (by default system directories will be checked). With this, passing of additional cmake options:

-DIGC_OPTION__LINK_KHRONOS_SPIRV_TRANSLATOR=ON -DIGC_OPTION__USE_KHRONOS_SPIRV_TRANSLATOR_IN_VC=ON -DIGC_OPTION__SPIRV_TRANSLATOR_MODE=Prebuilds

should solve problem with using of system installed translator instead of SPIRVDLL.

eero-t commented 3 years ago

Thanks! I assume modifications to Khronos version are done for some additional features. Are there any user-visible features that one would lose as result of using unmodified translator, or are these modification related e.g. just to how optimized the resulting code will be?

aus-intel commented 3 years ago

Patching was needed to support loading of translator as dynamic module. All other features are upstreamed to Khronos repository to corresponding release branches so no functional/performance issues expected.

eero-t commented 3 years ago

Great, thanks!

Looking at my earlier build log for latest "igc-1.0.8517" release, CMake options one gets with " -LH -DINSTALL_SPIRVDLL=0", are following:

...
// [Experimental] Enable link against Khronos SPIRV-LLVM-Translator, instead of legacy versions
IGC_OPTION__LINK_KHRONOS_SPIRV_TRANSLATOR:BOOL=OFF
...
// [Experimental] Enable usage of Khronos SPIRV-LLVM-Translator in Scalar Compiler
IGC_OPTION__USE_KHRONOS_SPIRV_TRANSLATOR_IN_SC:BOOL=OFF

// [Experimental] Enable usage of Khronos SPIRV-LLVM-Translator in Vector Compiler
IGC_OPTION__USE_KHRONOS_SPIRV_TRANSLATOR_IN_VC:BOOL=OFF
...

Testing enabling these [1], shows that in "igc-1.0.8517" release, "INSTALL_SPIRVDLL=0" option still needs to be specified, but even with that build still fails.

Build of "master" (i.e. with 7309126 commit) fails unless "GC_OPTION__USE_KHRONOS_SPIRV_TRANSLATOR_IN_SC" is "OFF" => upstream SPIRV does not work for that yet?

[1] And "IGC_OPTION__SPIRV_TRANSLATOR_MODE=Prebuilds", which is not shown by CMake "-LH" option for some reason.

aus-intel commented 3 years ago

Scalar part of IGC (that is controlled by IGC_OPTION__USE_KHRONOS_SPIRV_TRANSLATOR_IN_SC) is not ready to use Khronos version of translator now (there can be build/functional/perf regressions) and they use internal copy so no additional repos are required. To disable SPIRVDLL and remove requirement on SPIRV translator sources availability only VC part should be switched.

kv-sc commented 2 years ago

Finally this is done. We got rid of SPIRVDLL and it will no longer required here. Will update README.

ArchangeGabriel commented 2 years ago

@kvladimi Does that mean we should now set IGC_OPTION__USE_KHRONOS_SPIRV_TRANSLATOR_IN_SC=ON?

aus-intel commented 2 years ago

@ArchangeGabriel IGC_OPTION__USE_KHRONOS_SPIRV_TRANSLATOR_IN_SC=ON should not be used. This flag is for opencl (scalar) compiler. This component uses custom copy of translator that is located inside IGC code (AdaptorOCL/SPIRV) so SPIRVDLL is not required for this component in any mode (see https://github.com/intel/intel-graphics-compiler/issues/148#issuecomment-916994097).

Regarding vector compiler and SPIRVDLL current situation is the following. https://github.com/intel/intel-graphics-compiler/commit/3fde0acae81b48b5225e1760e1feb9ae44ccdbb2 removes SPIRVDLL from build completely. After this commit khronos version is always used by this component. Both configurations with sources of translator and prebuilds are supported. No additional changes to cmake command is required (docs were updated too: https://github.com/intel/intel-graphics-compiler/commit/d5e0b30a6176dedfc9fc9f361d5681accaf424f2). However, this patch is not part of any release yet and before this commit (https://github.com/intel/intel-graphics-compiler/commit/3fde0acae81b48b5225e1760e1feb9ae44ccdbb2) you should follow instructions from https://github.com/intel/intel-graphics-compiler/issues/148#issuecomment-916228192 to manually switch VC from SPIRVDLL with cmake flags.