Unable to build using clang-cl

[Epixu]

Hi there, I've been struggling with building zlib with MSVC's Clang-CL (-A Win32 -T ClangCl for example) My speculation is, that the preprocessor gets confused somewhere, because under Clang-CL, both _MSC_VER and __clang__ are defined, while only the latter has the problematic include_next. I'm unable to pinpoint the exact place of failure, though, as I'm not really familiar with zlib's configuration and source code

[Mr-Clam]

Can you use a different CMake generator such as Ninja? We have a slightly different setup, but we are successfully compiling with an older version of clang-cl (version 10). In our case, we're cross-compiling for Windows ARM using cmake + ninja on Linux.

RC1021 looks like a resource compiler error. Is it using the Microsoft resource compiler? It can be pretty flakey and presumably wouldn't like the LLVM header where the error's occurring. I can see in our build logs that that zlib1.rc is being processed with: /opt/cmake-3.20.3-linux-x86_64/bin/cmake -E cmake_llvm_rc <root_path>/zlib/win32/zlib1.rc CMakeFiles/zlib.dir/win32/zlib1.res.pp /usr/bin/clang-cl-10 <...>. Maybe you can pass -D CMAKE_VERBOSE_MAKEFILE=1 to the cmake config step to make your build log more informative?

[Mr-Clam]

Additionally, if it is using the wrong resource compiler, you can try to specify explicitly via the RC environmental variable before invoking the CMake configuration step, or by specifying CMAKE_RC_COMPILER in your CMakeLists.txt.

[Epixu]

RC1021 looks like a resource compiler error. Is it using the Microsoft resource compiler? It can be pretty flakey and presumably wouldn't like the LLVM header where the error's occurring. I can see in our build logs that that zlib1.rc is being processed with: /opt/cmake-3.20.3-linux-x86_64/bin/cmake -E cmake_llvm_rc <root_path>/zlib/win32/zlib1.rc CMakeFiles/zlib.dir/win32/zlib1.res.pp /usr/bin/clang-cl-10 <...>. Maybe you can pass -D CMAKE_VERBOSE_MAKEFILE=1 to the cmake config step to make your build log more informative?

It seems your suggestion is correct, I've enabled verbose makefile logging

Any suggestions on how to circumvent the issue without switching to Ninja? Would you consider having these test cases in your workflow?

[Mr-Clam]

Any suggestions on how to circumvent the issue without switching to Ninja?

See how you get on explicitly specifying the resource compiler.

Would you consider having these test cases in your workflow?

I personally like to run tests where they exist because they sometimes catch problems (generally when I've misconfigured something!) and it saves time finding it this way, although I've never seen them fail for zlib. We do do a make test for zlib on the platforms where we natively compile, but we have to skip this when cross-compiling.

[Epixu]

Additionally, if it is using the wrong resource compiler, you can try to specify explicitly via the RC environmental variable before invoking the CMake configuration step, or by specifying CMAKE_RC_COMPILER in your CMakeLists.txt.

I followed your suggestion, specifying explicitly the path where llvm-rc.exe is located on the runner (the one shipped with the Clang-Cl used by visual studio):

 - name: Windows MSVC Win32 (clang-cl)
    os: windows-latest
    compiler: cl
    cmake-args: -A Win32 -T ClangCl -D CMAKE_RC_COMPILER="C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/Llvm/bin/llvm-rc.exe"
 - name: Windows MSVC Win64 (clang-cl)
    os: windows-latest
    compiler: cl
    cmake-args: -A x64 -T ClangCl -D CMAKE_RC_COMPILER="C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/Llvm/bin/llvm-rc.exe"

But it doesn't seem to reflect at all in the log, it continues to use C:\Program Files (x86)\Windows Kits\10\bin\10.0.20348.0\x64\rc.exe instead, as before

[Epixu]

Also, it is important to note, that I'm actually able to build zlib locally on my PC, the problem seems to be occuring only on the github runners for some reason. For context, all my other projects build fine, both locally and remotely, using the conventional -T ClangCL configuration

[Mr-Clam]

Sorry, I'm speculating: perhaps it's hardcoded behaviour for MSBuild with VS project files? I can understand that you might not want to install ninja, but can you try the NMake generator instead, which comes with VS2022?

[Epixu]

I suppose Ninja has to be the solution, since locally Visual Studio seems to be using just that

[Mr-Clam]

I like Ninja: it's a single small binary, works the same way on all of our platforms and builds quickly. But if this is a problem related to using VS project files, NMake might also work and doesn't require installing anything. Just add -G "NMake Makefiles" to your cmake configure command.

[Epixu]

Now I'm trying to configure using Ninja, and it's just blatantly saying to me that clang-cl.exe is not able to compile a simple test program I'm running VsDevCmd.bat, but still not working

[Mr-Clam]

Does setting CMAKE_C_COMPILER fix this? And possibly CMAKE_CXX_COMPILER (is there an C++ in zlib?) and CMAKE_LINKER (or is CMAKE_C_LINK_EXECUTABLE?).

[Epixu]

@Mr-Clam thank you very much for your time, I learned a great deal of stuff! I was able to finally configure Ninja on the Github runner

For any lurkers about, here's how I managed to do it:

  - name: Windows MSVC Win32 (clang-cl)
    os: windows-latest
    compiler: clang-cl
    cmake-args: -G "Ninja" -DCMAKE_C_COMPILER=clang-cl -DCMAKE_C_FLAGS=-m32 -DCMAKE_MT="C:/Program Files (x86)/Windows Kits/10/bin/10.0.19041.0/x86/mt.exe"
    vcvars: cmd /c "C:/Program Files/Microsoft Visual Studio/2022/Enterprise/Common7/Tools/VsDevCmd.bat" -arch=x86 &&
  - name: Windows MSVC Win64 (clang-cl)
    os: windows-latest
    compiler: clang-cl
    cmake-args: -G "Ninja" -DCMAKE_C_COMPILER=clang-cl -DCMAKE_C_FLAGS=-m64 -DCMAKE_MT="C:/Program Files (x86)/Windows Kits/10/bin/10.0.19041.0/x64/mt.exe"
    vcvars: cmd /c "C:/Program Files/Microsoft Visual Studio/2022/Enterprise/Common7/Tools/VsDevCmd.bat" -arch=x64 &&

(here's the full script)

Conclusions:

1. Avoid the Visual Studio generators, if you have *.rc files - seems the resource compiler is hard-coded and can't be overridden.
2. You have to manually setup the environment in cmd, by calling VsDevCmd.bat with the appropriate arguments
3. CMAKE_MT must be manually specified when using Ninja, even after VsDevCmd.bat, as it is also for some reason hard-coded to not use the one provided with clang-cl - you have to manually point to the one coming from the Windows Kit

I have absolutely no idea why this has to be so hard, and not fully automated at this point. Do you have suggestions on where to file some issues? cracks knuckles