search

msys2 / MINGW-packages

Package scripts for MinGW-w64 targets to build under MSYS2.
https://packages.msys2.org
BSD 3-Clause "New" or "Revised" License
2.29k stars 1.22k forks source link

Request build libclc within llvm-project #11600

Closed lygstate closed 2 years ago

pal1000 commented 2 years ago

LLVM SPIRV translator should be built first as libclc SPIRV targets depend on it.

lygstate commented 2 years ago

Yeap, that's right

lygstate commented 2 years ago

LLVM SPIRV translator should built along with llvm libclc can be built separately, but I'll suggest built them at a single shot


# Building LLVM
git clone -b release/14.x --depth=1 \
https://github.com/llvm/llvm-project llvm-project

git clone -b llvm_release_140 --depth=1 \
https://github.com/KhronosGroup/SPIRV-LLVM-Translator llvm-project/llvm/projects/SPIRV-LLVM-Translator

mkdir llvm-project/build
pushd llvm-project/build
cmake ../llvm \
-DCMAKE_TOOLCHAIN_FILE=$CMAKE_TOOLCHAIN_MINGW_PATH \
-DCMAKE_INSTALL_PREFIX=/usr/x86_64-w64-mingw32/ \
-GNinja -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_CROSSCOMPILING=1 \
-DLLVM_ENABLE_RTTI=ON \
-DCROSS_TOOLCHAIN_FLAGS_NATIVE=-DLLVM_EXTERNAL_SPIRV_HEADERS_SOURCE_DIR=$PWD/../../SPIRV-Tools/external/SPIRV-Headers \
-DLLVM_EXTERNAL_SPIRV_HEADERS_SOURCE_DIR=$PWD/../../SPIRV-Tools/external/SPIRV-Headers \
-DLLVM_ENABLE_PROJECTS="clang;lld" \
-DLLVM_TARGETS_TO_BUILD="AMDGPU;X86" \
-DLLVM_OPTIMIZED_TABLEGEN=TRUE \
-DLLVM_ENABLE_ASSERTIONS=TRUE \
-DLLVM_INCLUDE_UTILS=OFF \
-DLLVM_INCLUDE_RUNTIMES=OFF \
-DLLVM_INCLUDE_TESTS=OFF \
-DLLVM_INCLUDE_EXAMPLES=OFF \
-DLLVM_INCLUDE_GO_TESTS=OFF \
-DLLVM_INCLUDE_BENCHMARKS=OFF \
-DLLVM_BUILD_LLVM_C_DYLIB=OFF \
-DLLVM_ENABLE_DIA_SDK=OFF \
-DCLANG_BUILD_TOOLS=ON \
-DLLVM_SPIRV_INCLUDE_TESTS=OFF

ninja install
popd

# Building libclc
mkdir llvm-project/build-libclc
pushd llvm-project/build-libclc
cmake ../libclc \
-DCMAKE_TOOLCHAIN_FILE=$CMAKE_TOOLCHAIN_MINGW_PATH \
-DCMAKE_INSTALL_PREFIX=/usr/x86_64-w64-mingw32/ \
-GNinja -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_CROSSCOMPILING=1 \
-DCMAKE_POLICY_DEFAULT_CMP0091=NEW \
-DCMAKE_CXX_FLAGS="-m64" \
-DLLVM_CONFIG="/usr/x86_64-w64-mingw32/bin/llvm-config" \
-DLLVM_CLANG="/usr/x86_64-w64-mingw32/bin/clang" \
-DLLVM_AS="/usr/x86_64-w64-mingw32/bin/llvm-as" \
-DLLVM_LINK="/usr/x86_64-w64-mingw32/bin/llvm-link" \
-DLLVM_OPT="/usr/x86_64-w64-mingw32/bin/opt" \
-DLLVM_SPIRV="/usr/x86_64-w64-mingw32/bin/llvm-spirv" \
-DLIBCLC_TARGETS_TO_BUILD="spirv-mesa3d-;spirv64-mesa3d-"
Biswa96 commented 2 years ago

Hi @lygstate, I have added a pull request to add the package. Would you like to test the packages from GitHub Actions artifacts of that pull request? The test packages can be downloaded from here https://github.com/msys2/MINGW-packages/actions/runs/2286687211. Follow these steps:

  1. Scroll down and download the correct artifact. It'll be a zip file.
  2. Extract the package (.tar.zst) from zip file.
  3. Install it with pacman -U the-package-name.tar.zst command.
lygstate commented 2 years ago 
Program symbols-check.py found: YES (C:/CI-Tools/msys64/mingw64/bin/python.exe C:/work/xemu/xemu-opengl/mesa/bin/symbols-check.py)Program install_megadrivers.py found: YES (C:/CI-Tools/msys64/mingw64/bin/python.exe C:/work/xemu/xemu-opengl/mesa/bin/install_megadrivers.py)
Program xmllint found: YES (C:\CI-Tools\msys64\mingw64\bin/xmllint.EXE)
Has header "sys/time.h" : YES 
../../src/util/meson.build:377: WARNING: Project targeting '>= 0.53' but tried to use feature introduced in '0.55.0': "test" keyword argument "protocol" value "gtest".
../../src/mapi/shared-glapi/meson.build:77: WARNING: Project targeting '>= 0.53' but tried to use feature introduced in '0.55.0': 
"test" keyword argument "protocol" value "gtest".
Program zstd found: YES (C:\CI-Tools\msys64\mingw64\bin/zstd.EXE)

../../src/compiler/nir/meson.build:328:8: ERROR: File /mingw64/share/clc/spirv-mesa3d-.spv does not exist.

A full log can be found at C:/work/xemu/xemu-opengl/mesa/build/windows-mingw64/meson-logs/meson-log.txt
Biswa96 commented 2 years ago

../../src/compiler/nir/meson.build:328:8: ERROR: File /mingw64/share/clc/spirv-mesa3d-.spv does not exist.

That file exists in libclc package. Here is the output from pacman file search:

$ pacman -F /mingw64/share/clc/spirv-mesa3d-.spv
mingw64/share/clc/spirv-mesa3d-.spv is owned by mingw64/mingw-w64-x86_64-libclc 14.0.3-1
lygstate commented 2 years ago

That's the cause, I am building with /mingw64/bin/meson that doens't support for msys2 style's PATH..

lygstate commented 2 years ago

The installed pc file need change from

includedir=/mingw64/include
libexecdir=/mingw64/share/clc

Name: libclc
Description: Library requirements of the OpenCL C programming language
Version: 0.2.0
Cflags: -I${includedir}
Libs: -L${libexecdir}

to

prefix=/mingw64
includedir=${prefix}/include
libexecdir=${prefix}/share/clc

Name: libclc
Description: Library requirements of the OpenCL C programming language
Version: 0.2.0
Cflags: -I${includedir}
Libs: -L${libexecdir}

to fixes the issue

Biswa96 commented 2 years ago

That pkgconfig file looks good to me. How can I reproduce the issue?

lygstate commented 2 years ago

That pkgconfig file looks good to me. How can I reproduce the issue?

Maybe because I installed pkgconf ? not pkg-config?

Biswa96 commented 2 years ago

pkgconf is the new one and replaces pkg-config.

Biswa96 commented 2 years ago

If I enable microsoft-clc option in mesa I see different error as following:

Checking for function "mmap" : NO
src/compiler/nir/meson.build:345:6: ERROR: Problem encountered: mmap required for dynamic libCLC loading
lygstate commented 2 years ago

clone this one https://gitlab.freedesktop.org/mesa/mesa.git

main branch with

export GALLIUM_ST="
      -D microsoft-clc=enabled
      -D static-libclc=all
      -D shared-llvm=enabled
"

meson \
  ../.. \
  -Dbuildtype=debug \
  -Dshared-glapi=enabled \
  $GALLIUM_ST \
  -Dosmesa=true \
  "-Dgallium-drivers=swrast,d3d12,zink" \
  "-Dvulkan-drivers=swrast,amd,microsoft-experimental" \
  -Dspirv-to-dxil=true \
  -Dbuild-tests=true \
  -Dcpp_rtti=false
lygstate commented 2 years ago

sorry, the config should be

prefix=/mingw64
includedir=${prefix}/include
libexecdir=${prefix}/share/clc

Name: libclc
Description: Library requirements of the OpenCL C programming language
Version: 0.2.0
Cflags: -I${includedir}
Libs: -L${libexecdir}
Biswa96 commented 2 years ago

Thanks. I can reproduce the issue. I shall try to upstream the changes first. I'll let you know.

Biswa96 commented 2 years ago

@lygstate I have tried to compile mesa with your given command as above. But there may be some conflict between DirectX-Headers and mingw-w64 headers repository. Here is the error:

In file included from F:/msys64/ucrt64/include/wtypes.h:8,
                 from F:/msys64/ucrt64/include/winscard.h:10,
                 from F:/msys64/ucrt64/include/windows.h:97,
                 from F:/msys64/ucrt64/include/rpc.h:16,
                 from ../subprojects/DirectX-Headers-1.0/include/directx/d3d12.h:26,
                 from ../src/microsoft/clc/compute_test.cpp:28:
F:/msys64/ucrt64/include/rpcndr.h:15:2: error: #error incorrect <rpcndr.h> version. Use the header that matches with the
 MIDL compiler.
   15 | #error incorrect <rpcndr.h> version. Use the header that matches with the MIDL compiler.
      |  ^~~~~

Can we utilize mingw-w64 headers in mesa in that build configuration? I have seen your other posts in that DirectX-Headers repository. It is possible to add missing symbols in dxd headers in mingw-w64. Feel free to create a discussion thread here or in Discord.

lygstate commented 2 years ago

@lygstate I have tried to compile mesa with your given command as above. But there may be some conflict between DirectX-Headers and mingw-w64 headers repository. Here is the error:

In file included from F:/msys64/ucrt64/include/wtypes.h:8,
                 from F:/msys64/ucrt64/include/winscard.h:10,
                 from F:/msys64/ucrt64/include/windows.h:97,
                 from F:/msys64/ucrt64/include/rpc.h:16,
                 from ../subprojects/DirectX-Headers-1.0/include/directx/d3d12.h:26,
                 from ../src/microsoft/clc/compute_test.cpp:28:
F:/msys64/ucrt64/include/rpcndr.h:15:2: error: #error incorrect <rpcndr.h> version. Use the header that matches with the
 MIDL compiler.
   15 | #error incorrect <rpcndr.h> version. Use the header that matches with the MIDL compiler.
      |  ^~~~~

Can we utilize mingw-w64 headers in mesa in that build configuration? I have seen your other posts in that DirectX-Headers repository. It is possible to add missing symbols in dxd headers in mingw-w64. Feel free to create a discussion thread here or in Discord.

Sould use my fork, official mesa doesn't support that yet, patches not merged yet. The directx-headers also should use my fork, https://github.com/lygstate/DirectX-Headers, otherwise it's won't running

Biswa96 commented 2 years ago

I have compiled mesa with your provided command using mingw-w64. The later one need some additions. But so far mesa only have two types compiler errors

  1. aggregating return value from d3d12 interfaces, similar as you have reported in directx-headers repository.
  2. __uuidof does not work in some cases but may be fixed in mingw-w64.

And some missing headers & libraries in mesa but not so fatal.