pschneider1968
commented
2 years ago Closed pschneider1968 closed 2 years ago
pschneider1968
commented
2 years ago 
whelanh
commented
2 years ago When I try and build a profile-build on Windows using MSYS2 MINGW64 and Clang using this modified Makefile, it builds the instrumented version fine and runs the benchmark, but then fails to build the "optimized executable..." version with the following error message:
llvm-profdata merge -output=stockfish.profdata *.profraw warning: default.profraw: malformed instrumentation profile data error: no profile can be merged
It does produce a file called default.profraw, but apparently can't merge it.
It does build fine if I use the make -j build command instead of make -j profile-build. But I am not sure if the build version is slower than a profile-build. The straight "make -j build" executable is slightly slower than the gcc compiled version (my cpu is AMD 3960x).
pschneider1968
commented
2 years ago I could not reproduce this error message with the modified Makefile, in my MSYS2/Clang env, the optimized executable builds and runs fine. I have Clang and LLVM version 13.0.0-5 installed, which are the most recent ones. Maybe you have an older version installed which exhibits this problem?
Could you please post the output of:
1) pacman -Q | grep mingw-w64-clang | sort 2) ldd stockfish.exe 3) echo $PATH
Just to be sure, run "make clean" and check that you don't have older profile directory/data around in your build directory, and try to see whether the problem persists.
whelanh
commented
2 years ago Thank you for responding Peter. Running the commands you specified from my MSYS2 MINGW64 terminal (should I be running this in MSYS2 CLANG terminal instead?), I get:
$ pacman -Q | grep mingw-w64-clang | sort mingw-w64-clang-x86_64-compiler-rt 13.0.0-5
if I run: $ pacman -Q | grep clang | sort mingw-w64-clang-x86_64-compiler-rt 13.0.0-5 mingw-w64-ucrt-x86_64-clang 13.0.0-5 mingw-w64-x86_64-clang 13.0.0-5
Also: $ pacman -Q | grep llvm | sort mingw-w64-ucrt-x86_64-llvm 13.0.0-5 mingw-w64-x86_64-llvm 13.0.0-5
$ ldd stockfish.exe ldd: stockfish.exe: Bad file descriptor
$ echo $PATH /mingw64/bin:/usr/local/bin:/usr/bin:/bin:/c/Windows/System32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1.0/:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl
If I run the 3 commands you specified in my MSYS2 Clang terminal, I get the same outputs as above. If I run "make -j build ARCH=x86-64-avx2 COMP=clang" in my MSYS2 Clang terminal I get different errors including "make: [Makefile:962: .depend] Error 127 (ignored)" and "make[1]: clang++: No such file or directory"
ppigazzini
commented
2 years ago You need to use one of the MSYS2-CLANG 64 bit launchers to have the right environment for clang 64 bit, see https://www.msys2.org/docs/environments/
pschneider1968
commented
2 years ago Hmm, let's try to sort this out...
1) In your original issue you wrote: I try and build a profile-build on Windows using MSYS2 MINGW64 and Clang
Does that mean that you tried this Clang build using your MSYS2 MINGW64 terminal? In the MINGW64 terminal session, none of the Clang/LLVM tools should be in the path. You should do the Clang build in the CLANG64 terminal. The MINGW64 terminal session is intended to be used for the GNU/GCC toolchain (if you have that installed).
2) If you have installed the Clang toolchain, the "pacman -Q" command should output a long list of components that are installed. That you only have the compiler-rt installed looks very strange... Did you post only part of output? Please retry using the MSYS2 terminal.
3) That ldd command was meant to be run inside the src directory where you performed the build. There, the built stockfish.exe should still be around, and ldd would show the linked in libraries of the binary. Could you please recheck?
4) Your last paragraph is also strange. Does that mean you can't call the clang++ compiler from within the CLANG64 terminal?
Please try the commands in bold from the CLANG64 terminal and post the results/compare with the output from my env:
Peter@bolide CLANG64 ~/repos/pschneider1968/Stockfish/src $ clang++ --version clang version 13.0.0 Target: x86_64-w64-windows-gnu Thread model: posix InstalledDir: C:/msys64/clang64/bin
Peter@bolide CLANG64 ~/repos/pschneider1968/Stockfish/src $ ld.lld --version LLD 13.0.0 (compatible with GNU linkers)
Peter@bolide CLANG64 ~/repos/pschneider1968/Stockfish/src $ llvm-profdata.exe llvm-profdata.exe: No command specified! USAGE: llvm-profdata.exe <merge|show|overlap> [args...]
Peter@bolide CLANG64 ~/repos/pschneider1968/Stockfish/src $ which clang++ /clang64/bin/clang++
Peter@bolide CLANG64 ~/repos/pschneider1968/Stockfish/src $ which ld.lld /clang64/bin/ld.lld
Peter@bolide CLANG64 ~/repos/pschneider1968/Stockfish/src $ which llvm-profdata.exe /clang64/bin/llvm-profdata.exe
whelanh
commented
2 years ago Thank you very much Peter. You solved my problem. I had assumed since I had installed clang in my MSYS2 MINGW64 terminal it was available to the MSYS2 Clang terminal (but it wasn't). When I installed clang ("pacman -S mingw-w64-clang-x86_64-clang") in the my MSYS2 Clang terminal and did a clean compile, everything worked perfectly and it compiled the optimized version. I also get the same responses to the clang++ --version, which ld.lld etc commands that you do. In a very quick test using bench_parallel.sh I am now getting about a 4% speed-up using the clang compilation. Thank you so much for your patience.
pschneider1968
commented
2 years ago Sounds good! I'm glad that we could sort this out, and I'm happy that I did not introduce a regression with my Makefile changes š
BTW: if you do development work with MSYS2, you might want to install the whole package group for your toolchain, not just only the compiler itself. Please see the MSYS2 package group page
The bare minimum you should install is: base-devel expect git, then either (or both) the toolchain package groups
_mingw-w64-clang-x8664-toolchain for the Clang/LLVM toolchain, and/or
_mingw-w64-x8664-toolchain for the GNU/GCC toolchain.
To maintain the installation with pacman, I always use the MSYS2 terminal session only (but I'm not sure whether that makes a difference or not).
HTH, Peter
whelanh
commented
2 years ago Very helpful! Will install the toolchain. ...and thank you for your contribution to Stockfish.
Kindest regards, Hugh Whelan
On Wed, Jan 12, 2022 at 5:29 PM Peter Schneider @.***> wrote:
Sounds good! I'm glad that we could sort this out, and I'm happy that I did not introduce a regression with my Makefile changes š
BTW: if you do development work with MSYS2, you might want to install the whole package group for your toolchain, not just only the compiler itself. Please see the MSYS2 package group page https://packages.msys2.org/group/
The bare minimum you should install is: base-devel expect git, then either (or both) the toolchain package groups
mingw-w64-clang-x86_64-toolchain for the Clang/LLVM toolchain, and/or
mingw-w64-x86_64-toolchain
HTH, Peter
ā Reply to this email directly, view it on GitHub https://github.com/official-stockfish/Stockfish/issues/3872#issuecomment-1011509100, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC2JOXMHTOAZB43KI2GK7U3UVX6FPANCNFSM5K25EGPQ . You are receiving this because you commented.Message ID: @.***>
pschneider1968
commented
2 years ago TBH this is my very first attempt to contribute something to Stockfish, but I would be happy to be able to contribute more in the future, as I (hopefully) learn more about it. I'm very thankful to the Stockfish devs, that such a great project exists and is open source and free software! I use it all the time to analyze my (lost! š) OTB chess games, and although SF always tells me I'm an idiot and shouldn't play chess, I still love it!
ppigazzini
commented
2 years ago I had assumed since I had installed clang in my MSYS2 MINGW64 terminal it was available to the MSYS2 Clang terminal (but it wasn't).
Only to point out the right nomenclature.
C:\tools\msys64\msys2_shell.cmd -clang64 ("MSYS2 MinGW Clang 64-bit" shourtcut)C:\tools\msys64\clang64.exeStarting one of the clang 64 bit launchers you get the mintty terminal, using the bash shell, with the environment variables set to find the clang 64 bit toolchain.
I have encountered two build problems using Windows 10, MSYS2 and Clang 13 trying to build current HEAD of Stockfish. (Compiling with GCC 11.2 works fine, though)
1) Linkage fails due to use of -latomic 2) When I work around this problem (either by changing the Makefile, or by additionally specifying "RTLIB=compiler-rt"), the build succeeds, but the resulting binary cannot run outside of MSYS/Clang env because it is NOT statically linked. It gives the error message, that it can't find libc++.dll
More details:
1.
After reading through the LLVM/Clang docs at https://clang.llvm.org/docs/Toolchain.html#compiler-runtime and reviewing the Makefile, I additionally specified RTLIB=compiler-rt (leading to not adding "-latomic" to the LDFLAGS) and then the build succeeds. In the CLANG64 env in MSYS2, libatomic.a is not available. It is only installed in the MINGW64 env with the GNU toolchain and GCC. Is this supposed to work this way? Then I think, at least it should be documented in the build instructions for Windows. What also confused me is that the RTLIB variable is not used anymore in the Makefile beyond that. I.e. it is not passed "-DLIBCXX_USE_COMPILER_RT=YES" to the compiler, as the LLVM Clang docs state. Or is this the default anyway? Sorry if this question is dumb - I'm not a C or C++ dev...
2.
I make with
and the build succeeds, however the binary only runs in the CLANG64 env, due to not linking -static:
When e.g. I try to run the binary from a new CMD.exe command prompt, it displays this error message:
I think on Windows, it needs to be statically linked. When I add these lines to the Makefile in the Clang section, the built executable works fine: