m-ab-s / media-autobuild_suite

This Windows Batchscript helps setup a Mingw-w64 compiler environment for building ffmpeg and other media tools under Windows.
GNU General Public License v3.0
1.49k stars 257 forks source link

[mingw64] glslang fails to build because of missing libHLSL #2558

Closed twokilohertz closed 7 months ago

twokilohertz commented 7 months ago

I'm running the suite trying to build 64-bit static & shared FFmpeg however I'm repeatedly running into this issue where glslang will fail to build because "libHLSL.a" is missing. I've attached some logs from MABS & the glslang-git directory. They don't seem to be too helpful, however. I can't work out why libHLSL.a is not being generated.

[...]
┌ vulkan-loader git  .................................. [Recently updated]
├ Installing uasm... done
├ Running dependencies...
├ Installing Vulkan-Headers...
├ Running cmake...
├ Running build...
├ Running install...
├ Building Vulkan-Loader...
├ Running cmake...
├ Running build...
├ Running install...
├ Stripping...
└ vulkan-loader-git  ........................................... [Updated]
  Running git clone for SPIRV-Cross...
  Running git update for SPIRV-Cross...
┌ SPIRV-Cross git  .................................... [Recently updated]
├ Running meson...
├ Running build...
├ Running install...
├ Stripping...
└ SPIRV-Cross-git  ............................................. [Updated]
  Running git clone for glslang...
  Running git update for glslang...
┌ glslang git  ........................................ [Recently updated]
├ Installing python... done
├ Running dependencies...
├ Running cmake...
├ Running build...
├ Running install...
├ /local64/lib/libHLSL.a  .................................... [Not found]
└ glslang-git  .................................................. [Failed]

Try deleting '/build/glslang-git' and start the script again.
If you are sure there are no dependencies, <Enter> to continue building.

Close this window if you wish to stop building.

issue.zip

twokilohertz commented 7 months ago

I've attached my INI file & ffmpeg/mpv build configs here.

build.zip

diegocr commented 7 months ago

Until it's fixed...

glslang_extra.sh

_pre_ninja(){
    cd "${REPO_DIR}" && git reset --hard 920a1424c998c4c09f35cdf80ebb482d626e655f
}
twokilohertz commented 7 months ago

Until it's fixed...

glslang_extra.sh

_pre_ninja(){
    cd "${REPO_DIR}" && git reset --hard 920a1424c998c4c09f35cdf80ebb482d626e655f
}

Thanks for the fix! I didn't know that there was a broken commit upstream. I'll try this out now. :)

Biswa96 commented 7 months ago

The issue is in m-ab-s project. Upstream removed OGLCompiler and HLSL stub libraries in this commit https://github.com/KhronosGroup/glslang/commit/6be56e45e574b375d759b89dad35f780bbd4792f

The following patch in this repository may fix the issue, though I have not tested it.

--- a/build/media-suite_compile.sh
+++ b/build/media-suite_compile.sh
@@ -2003,7 +2003,7 @@ if { { [[ $mpv != n ]] && ! mpv_disabled libplacebo; } ||
     do_checkIfExist
 fi

-_check=(lib{glslang,OSDependent,HLSL,OGLCompiler,SPVRemapper}.a
+_check=(lib{glslang,OSDependent,SPVRemapper}.a
         libSPIRV{,-Tools{,-opt,-link,-reduce}}.a glslang/SPIRV/GlslangToSpv.h)
 if { { [[ $mpv != n ]]  && ! mpv_disabled libplacebo; } ||
      { [[ $ffmpeg != no ]] && enabled_any libplacebo libglslang; } } &&
twokilohertz commented 7 months ago

Gotcha. That'd be why then. I'll try out reverting the commit first, wait for this rebuild to finish (fail) and then I'll give that patch a go.

Thank you for the input. :)

twokilohertz commented 7 months ago

Update: tried the glslang_extra.sh pre-ninja method but unfortunately didn't solve the issue. I shall try your patch @Biswa96.

Likely error (tail of the failed operation logfile):
[18/21] Linking CXX static library libshaderc\libshaderc_combined.a
[19/21] Linking CXX static library glslc\libglslc.a
[20/21] Building CXX object glslc/CMakeFiles/glslc_exe.dir/src/main.cc.o
FAILED: glslc/CMakeFiles/glslc_exe.dir/src/main.cc.o
C:\build\MABS\msys64\mingw64\bin\ccache.exe  g++  -IC:/build/MABS/build/shaderc-git/build-64bit/glslc/.. -IC:/build/MABS/build/shaderc-git/third_party/spirv-tools/include -IC:/build/MABS/build/shaderc-git/third_party/glslang -IC:/build/MABS/build/shaderc-git/libshaderc_util/include -IC:/build/MABS/build/shaderc-git/libshaderc/include -D_FORTIFY_SOURCE=2 -fstack-protector-strong -mtune=generic -O2 -pipe -D__USE_MINGW_ANSI_STDIO=1 -Wimplicit-fallthrough -O3 -DNDEBUG -std=gnu++17 -Wextra-semi -Wall -fvisibility=hidden -MD -MT glslc/CMakeFiles/glslc_exe.dir/src/main.cc.o -MF glslc\CMakeFiles\glslc_exe.dir\src\main.cc.o.d -o glslc/CMakeFiles/glslc_exe.dir/src/main.cc.o -c C:/build/MABS/build/shaderc-git/glslc/src/main.cc
C:/build/MABS/build/shaderc-git/glslc/src/main.cc:208:10: fatal error: build-version.inc: No such file or directory
  208 | #include "build-version.inc"
      |          ^~~~~~~~~~~~~~~~~~~
compilation terminated.
ninja: build stopped: subcommand failed.
build failed. Check C:/build/MABS/build/shaderc-git/build-64bit/ab-suite.build.log
This is required for other packages, so this script will exit.
  Creating diagnostics file...

Attach C:\build\MABS\build\logs.zip to the GitHub issue.
Make sure the suite is up-to-date before reporting an issue. It might've been fixed already.

Try running the build again at a later time.
diegocr commented 7 months ago

That seems a problem with shaderc, no longer with glslang, fwiw :)

twokilohertz commented 7 months ago

That seems a problem with shaderc, no longer with glslang, fwiw :)

Ah, yes. I think you're right...

Mr-Z-2697 commented 7 months ago

And ffmpeg is still looking for those libraries in configure.

twokilohertz commented 7 months ago

The issue is in m-ab-s project. Upstream removed OGLCompiler and HLSL stub libraries in this commit KhronosGroup/glslang@6be56e4

The following patch in this repository may fix the issue, though I have not tested it.

--- a/build/media-suite_compile.sh
+++ b/build/media-suite_compile.sh
@@ -2003,7 +2003,7 @@ if { { [[ $mpv != n ]] && ! mpv_disabled libplacebo; } ||
     do_checkIfExist
 fi

-_check=(lib{glslang,OSDependent,HLSL,OGLCompiler,SPVRemapper}.a
+_check=(lib{glslang,OSDependent,SPVRemapper}.a
         libSPIRV{,-Tools{,-opt,-link,-reduce}}.a glslang/SPIRV/GlslangToSpv.h)
 if { { [[ $mpv != n ]]  && ! mpv_disabled libplacebo; } ||
      { [[ $ffmpeg != no ]] && enabled_any libplacebo libglslang; } } &&

This patch works for fixing glslang's build issue. The build issue relating to shaderc is different. In theory, Python should be invoked and the "build-version.inc" file should be generated. I wonder if anyone else can replicate this.

twokilohertz commented 7 months ago

I'll close this issue now as the problem has been solved. A separate issue for shaderc is appropriate.

Biswa96 commented 7 months ago

The issue has not been fixed yet in this repository. I would suggest to keep this issue open because it may help maintainers to track it.

twokilohertz commented 7 months ago

My bad. I saw Hyrda's commit and my brain just assumed it was a submitted PR to this repo. Only just woke up :^)

hydra3333 commented 7 months ago

Sorry, I only mentioned the link in temporary a forked repo.

assy1204 commented 7 months ago

A new build with the latest version also gives an error.

Running git clone for glslang... Running git update for glslang... ┌ glslang git ........................................ [Recently updated] ├ Installing python... done ├ Running dependencies... ├ Running cmake... ├ Running build... ├ Running install... ├ /local64/lib/libSPIRV-Tools.a ............................. [Not found] └ glslang-git .................................................. [Failed]

Try deleting '/build/glslang-git' and start the script again. If you are sure there are no dependencies, to continue building.

Close this window if you wish to stop building.

twokilohertz commented 7 months ago

Yep. I got that too. It's another thing that's changed in the glslang repo. I just rolled the commit back to previous temporarily. This is the offending commit.

Biswa96 commented 7 months ago

Just a guess. Please try the following change (a comma is removed).

--- a/build/media-suite_compile.sh
+++ b/build/media-suite_compile.sh
@@ -2004,7 +2004,7 @@ if { { [[ $mpv != n ]] && ! mpv_disabled libplacebo; } ||
 fi

 _check=(lib{glslang,OSDependent,SPVRemapper}.a
-        libSPIRV{,-Tools{,-opt,-link,-reduce}}.a glslang/SPIRV/GlslangToSpv.h)
+        libSPIRV{,-Tools{-opt,-link,-reduce}}.a glslang/SPIRV/GlslangToSpv.h)
 if { { [[ $mpv != n ]]  && ! mpv_disabled libplacebo; } ||
      { [[ $ffmpeg != no ]] && enabled_any libplacebo libglslang; } } &&
     do_vcs "$SOURCE_REPO_GLSLANG"; then
fumiichan commented 7 months ago

According to that commit, it will no longer build spirv-tools so maybe that patch will not work. Just tried earlier and yeah still failed:

15:41:08   Running git update for SPIRV-Cross...
15:41:09 SPIRV-Cross git  .................................. [Up-to-date]
15:41:11   Running git update for glslang...
15:41:13 ┌ glslang git  .............................. [Recently updated]
15:41:15 ├ Running uninstall...
15:41:15 ├ Running dependencies...
15:41:28 ├ Running cmake...
15:41:32 ├ Running build...
15:41:44 ├ Running install...
15:41:45 ├ /local64/lib/libSPIRV-Tools-opt.a  ............... [Not found]
15:41:45 └ glslang-git  ........................................ [Failed]

Try deleting '/build/glslang-git' and start the script again.
If you are sure there are no dependencies, <Enter> to continue building.
Biswa96 commented 7 months ago

According to that commit, it will no longer build spirv-tools so maybe that patch will not work.

But there are some spirv-tools libraries in CI log of glslang repository. You can remove the libSPIRV{,-Tools{,-opt,-link,-reduce}}.a part from the build/media-suite_compile.sh file.