search

official-stockfish / Stockfish

A free and strong UCI chess engine
https://stockfishchess.org/
GNU General Public License v3.0
11.33k stars 2.25k forks source link

LTO on windows #2769

Closed abdulbadii closed 4 years ago

abdulbadii commented 4 years ago

@snicolet please watch the latest update makefile! and watch the first author's comment and what one has warned: https://github.com/official-stockfish/Stockfish/issues/2731#issuecomment-647805424 Link Time Optimization, it works since gcc 4.5 but not on mingw under Windows. that is his noble forewarn and correctly resolve it for us:

    ifneq (,$(filter $(comp),gcc clang))
        CXXFLAGS += -flto
        LDFLAGS += $(CXXFLAGS)

    else ifeq ($(comp),mingw)
        ifeq ($(KERNEL),Linux)
            CXXFLAGS += -flto
            LDFLAGS += $(CXXFLAGS)
        endif

below is it, it's on -flto and indeed it outright fails on Windows as author said and proved a failure on my Windows system execution

    ifeq ($(comp),$(filter $(comp),gcc clang mingw))
        CXXFLAGS += -flto
        LDFLAGS += $(CXXFLAGS)

It only compiles without echoing any error

unaiic commented 4 years ago

Maybe we should revert the MinGW part of 11483fe6d942a4fee6fa272f72251d6b6d6d7454 until we find something better.

snicolet commented 4 years ago

@abdulbadii

If I may, I find it a little bit agressive that you give me orders, use sentences with exclamation marks, etc, because you feel that I don't review your makefile fast enough :-) This is a part-time hobby, don't forget.

stockchess commented 4 years ago

@abdulbadii

new makefile works perfectly with lto in windows with mingw, maybe you need to update your mingw

vondele commented 4 years ago

this needs an update with the version of gcc used, and the precise error message seen.

vondele commented 4 years ago

was also reported on fishcooking: https://groups.google.com/g/fishcooking/c/pCWZVmzdsOs

abdulbadii commented 4 years ago

@Snicolet I apologize, I admitted you are correct I really forget: This is our part-time hobby.

This's extremely so weird, updated to the latest:

$ gcc  --version
gcc.exe (Rev3, Built by MSYS2 project) 10.1.0
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE

and make option -flto build

$ make build ARCH=x86-64-modern
Config:
debug: 'no'
sanitize: 'no'
optimize: 'yes'
arch: 'x86_64'
bits: '64'
kernel: 'MINGW64_NT-10.0-14393'
os: 'Windows_NT'
prefetch: 'yes'
popcnt: 'yes'
sse: 'yes'
pext: 'no'

Flags:
CXX: g++
CXXFLAGS: -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT
LDFLAGS:  -flto -static

Testing config sanity. If this fails, try 'make help' ...

g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o benchmark.o benchmark.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o bitbase.o bitbase.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o bitboard.o bitboard.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o endgame.o endgame.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o evaluate.o evaluate.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o main.o main.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o material.o material.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o misc.o misc.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o movegen.o movegen.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o movepick.o movepick.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o pawns.o pawns.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o position.o position.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o psqt.o psqt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o search.o search.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o thread.o thread.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o timeman.o timeman.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o tt.o tt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o uci.o uci.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o ucioption.o ucioption.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o tune.o tune.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o tbprobe.o syzygy/tbprobe.cpp
g++ -o stockfish.exe benchmark.o bitbase.o bitboard.o endgame.o evaluate.o main.o material.o misc.o movegen.o movepick.o pawns.o position.o psqt.o search.o thread.o timeman.o tt.o uci.o ucioption.o tune.o tbprobe.o  -flto -static
user@HP_A10 /usr/src/Stockfish/src

$ ./stockfish.exe
Segmentation fault

old one

$ make -f Makefile_o build ARCH=x86-64-modern

Config:
debug: 'no'
sanitize: 'no'
optimize: 'yes'
arch: 'x86_64'
bits: '64'
kernel: 'MINGW64_NT-10.0-14393'
os: 'Windows_NT'
prefetch: 'yes'
popcnt: 'yes'
sse: 'yes'
pext: 'no'

Flags:
CXX: g++
CXXFLAGS: -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT
LDFLAGS:   -static

Testing config sanity. If this fails, try 'make help' ...

g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o benchmark.o benchmark.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o bitbase.o bitbase.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o bitboard.o bitboard.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o endgame.o endgame.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o evaluate.o evaluate.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o main.o main.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o material.o material.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o misc.o misc.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o movegen.o movegen.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o movepick.o movepick.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o pawns.o pawns.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o position.o position.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o psqt.o psqt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o search.o search.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o thread.o thread.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o timeman.o timeman.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o tt.o tt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o uci.o uci.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o ucioption.o ucioption.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o tune.o tune.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o tbprobe.o syzygy/tbprobe.cpp
g++ -o stockfish.exe benchmark.o bitbase.o bitboard.o endgame.o evaluate.o main.o material.o misc.o movegen.o movepick.o pawns.o position.o psqt.o search.o thread.o timeman.o tt.o uci.o ucioption.o tune.o tbprobe.o   -static
user@HP_A10 /usr/src/Stockfish/src

$ ./stockfish.exe
Stockfish 250620 64 POPCNT by T. Romstad, M. Costalba, J. Kiiski, G. Linscott
unaiic commented 4 years ago

@vondele IMHO it could be better to restore the previous mingw stuff and try to find a solution, leaving the rest of 11483fe as it was commited, at least ensuring no problems with windows users for now

vondele commented 4 years ago

I would first like us to understand this, before making new changes in a rush.

joergoster commented 4 years ago

@abdulbadii What happened to your LDFLAGS? Did you modify the makefile?

make build ARCH=x86-64-modern COMP=mingw -j3

Config:
debug: 'no'
sanitize: 'no'
optimize: 'yes'
arch: 'x86_64'
bits: '64'
kernel: 'MINGW64_NT-10.0-18363'
os: 'Windows_NT'
prefetch: 'yes'
popcnt: 'yes'
sse: 'yes'
pext: 'no'

Flags:
CXX: g++
CXXFLAGS: -Wall -Wcast-qual -fno-exceptions -std=c++11  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto
LDFLAGS:  -static -Wall -Wcast-qual -fno-exceptions -std=c++11  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto

Testing config sanity. If this fails, try 'make help' ...
stockchess commented 4 years ago

gcc gcc.exe (Rev3, Built by MSYS2 project) 10.1.0

command make -j profile-build ARCH=x86-64-bmi2 COMP=mingw

output

Config:
debug: 'no'
sanitize: 'no'
optimize: 'yes'
arch: 'x86_64'
bits: '64'
kernel: 'MINGW64_NT-10.0-17134'
os: 'Windows_NT'
prefetch: 'yes'
popcnt: 'yes'
sse: 'yes'
pext: 'yes'

Flags:
CXX: g++
CXXFLAGS: -Wall -Wcast-qual -fno-exceptions -std=c++11  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto
LDFLAGS:  -static -Wall -Wcast-qual -fno-exceptions -std=c++11  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto

Testing config sanity. If this fails, try 'make help' ...

Step 1/4. Building instrumented executable ...
make ARCH=x86-64-bmi2 COMP=mingw gcc-profile-make
make[1]: Entering directory '/home/S/stockfish/src'
make ARCH=x86-64-bmi2 COMP=mingw \
EXTRACXXFLAGS='-fprofile-generate' \
EXTRALDFLAGS='-lgcov' \
all
make[2]: Entering directory '/home/S/stockfish/src'
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o benchmark.o benchmark.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o bitbase.o bitbase.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o bitboard.o bitboard.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o endgame.o endgame.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o evaluate.o evaluate.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o main.o main.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o material.o material.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o misc.o misc.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o movegen.o movegen.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o movepick.o movepick.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o pawns.o pawns.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o position.o position.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o psqt.o psqt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o search.o search.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o thread.o thread.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o timeman.o timeman.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o tt.o tt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o uci.o uci.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o ucioption.o ucioption.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o tune.o tune.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o tbprobe.o syzygy/tbprobe.cpp
g++ -o stockfish.exe benchmark.o bitbase.o bitboard.o endgame.o evaluate.o main.o material.o misc.o movegen.o movepick.o pawns.o position.o psqt.o search.o thread.o timeman.o tt.o uci.o ucioption.o tune.o tbprobe.o -lgcov -static -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto
make[2]: Leaving directory '/home/S/stockfish/src'
make[1]: Leaving directory '/home/S/stockfish/src'

Step 2/4. Running benchmark for pgo-build ...
./stockfish.exe bench > /dev/null

Position: 1/47

Position: 2/47

Position: 3/47

Position: 4/47

Position: 5/47

Position: 6/47

Position: 7/47

Position: 8/47

Position: 9/47

Position: 10/47

Position: 11/47

Position: 12/47

Position: 13/47

Position: 14/47

Position: 15/47

Position: 16/47

Position: 17/47

Position: 18/47

Position: 19/47

Position: 20/47

Position: 21/47

Position: 22/47

Position: 23/47

Position: 24/47

Position: 25/47

Position: 26/47

Position: 27/47

Position: 28/47

Position: 29/47

Position: 30/47

Position: 31/47

Position: 32/47

Position: 33/47

Position: 34/47

Position: 35/47

Position: 36/47

Position: 37/47

Position: 38/47

Position: 39/47

Position: 40/47

Position: 41/47

Position: 42/47

Position: 43/47

Position: 44/47

Position: 45/47

Position: 46/47

Position: 47/47

===========================
Total time (ms) : 3286
Nodes searched  : 4789930
Nodes/second    : 1457678

Step 3/4. Building optimized executable ...
make ARCH=x86-64-bmi2 COMP=mingw objclean
make[1]: Entering directory '/home/S/stockfish/src'
make[1]: Leaving directory '/home/S/stockfish/src'
make ARCH=x86-64-bmi2 COMP=mingw gcc-profile-use
make[1]: Entering directory '/home/S/stockfish/src'
make ARCH=x86-64-bmi2 COMP=mingw \
EXTRACXXFLAGS='-fprofile-use -fno-peel-loops -fno-tracer' \
EXTRALDFLAGS='-lgcov' \
all
make[2]: Entering directory '/home/S/stockfish/src'
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o benchmark.o benchmark.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o bitbase.o bitbase.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o bitboard.o bitboard.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o endgame.o endgame.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o evaluate.o evaluate.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o main.o main.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o material.o material.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o misc.o misc.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o movegen.o movegen.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o movepick.o movepick.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o pawns.o pawns.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o position.o position.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o psqt.o psqt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o search.o search.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o thread.o thread.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o timeman.o timeman.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o tt.o tt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o uci.o uci.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o ucioption.o ucioption.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o tune.o tune.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o tbprobe.o syzygy/tbprobe.cpp
g++ -o stockfish.exe benchmark.o bitbase.o bitboard.o endgame.o evaluate.o main.o material.o misc.o movegen.o movepick.o pawns.o position.o psqt.o search.o thread.o timeman.o tt.o uci.o ucioption.o tune.o tbprobe.o -lgcov -static -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto
make[2]: Leaving directory '/home/S/stockfish/src'
make[1]: Leaving directory '/home/S/stockfish/src'

Step 4/4. Deleting profile data ...
make ARCH=x86-64-bmi2 COMP=mingw profileclean
make[1]: Entering directory '/home/S/stockfish/src'
make[1]: Leaving directory '/home/S/stockfish/src'
Rocky640 commented 4 years ago

here it is gcc version 6.1.0

gcc.exe (x86_64-posix-seh, Built by MinGW-W64 project) 6.1.0

builds which works with old master Flags: CXX: g++ CXXFLAGS: -Wall -Wcast-qual -fno-exceptions -std=c++11 -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT LDFLAGS: -static

latest build fails

Flags: CXX: g++ CXXFLAGS: -Wall -Wcast-qual -fno-exceptions -std=c++11 -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto LDFLAGS: -static -Wall -Wcast-qual -fno-exceptions -std=c++11 -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto

with the latest makefile, and removing > /dev/null I get the following

make[2]: Leaving directory /c/Users/Alain/Documents/GitHub/Stockfish/src' make[1]: Leaving directory/c/Users/Alain/Documents/GitHub/Stockfish/src'

Step 2/4. Running benchmark for pgo-build ... ./stockfish.exe bench make: *** [profile-build] Error 5

my install came from the fishtest portable exe. I installed this in January 2020

vondele commented 4 years ago

OK, thanks for the info. So gcc 6 doesn't work seemingly. https://www.gnu.org/software/gcc/gcc-6/ says it is no longer supported (was first released in 2016).

However, I can see an issue if that is what is part of our fishtest portable exe.

@ppigazzini do you have a suggestion/idea here?

LTO is a rather important optimization, a quick check here suggests it is currently about 7% speedup.

Rocky640 commented 4 years ago

here is some ideas to test the gcc version in a makefile https://stackoverflow.com/questions/5188267/checking-the-gcc-version-in-a-makefile

vondele commented 4 years ago

To be honest, I would rather see how we can drop supporting old compiler versions. If even the team developing it stops supporting it, why should we support it? Of course, it would mean that some devs have to upgrade, which is unfortunate, but as the 7% shows, has a benefit as well.

ppigazzini commented 4 years ago

@vondele I already built and tested a new windows-portable archive with gcc 8.x and python3. I need a host where upload the archive and get a link to be used in a bat script (GDrive is out). Unfortunately Gary isn't the owner of the S3 bucket where is stored the original window-portable archive :( Suggestions are welcome.

vondele commented 4 years ago

@ppigazzini options that would come to mind would be to

I somehow prefer the first, we're presumably talking about a handful of downloads a week, the server should have no problem with that... but maybe I overlooked something?

mstembera commented 4 years ago

FYI I can also confirm that the addition of -flto to the mingw compile does break

make profile-build ARCH=x86-64-bmi2 COMP=mingw
...
Step 2/4. Running benchmark for pgo-build ...
./stockfish.exe bench > /dev/null
make: *** [profile-build] Error 5

on Windows using mingw and gcc 8.1.0 In addition the non profile build does compile but strangely produces no output when I run bench. Removing either the COMP=mingw or the -flto fixes the problems. Someone in the past has figured out that the -flto and -static options don't work together on this platform. Indeed removing -static also fixes the problem but then of course we don't have a self contained binary.

FauziAkram commented 4 years ago

Is there a solution to this? It's been 2 days I can't compile any patch, and I have alot of tests in my pipeline xD

Rocky640 commented 4 years ago

@FauziAkram if you only need a bench number, to compile you can use previous \src\makefile, of simply remove those lines from the new one


### 3.8 Link Time Optimization
### This is a mix of compile and link time options because the lto link phase
### needs access to the optimization flags.
ifeq ($(optimize),yes)
ifeq ($(debug), no)
    ifeq ($(comp),$(filter $(comp),gcc clang mingw))
        CXXFLAGS += -flto
        LDFLAGS += $(CXXFLAGS)
    endif
endif
endif
MichaelB7 commented 4 years ago

I’m not sure what we gained by fixing something that was not broken , and it’s now broken. My mingw is updated. I just reverted the makefile - but flto was working fine for me prior to the patch.

vondele commented 4 years ago

@FauziAkram another workaround without changing the sources is to add optimize=no to the make command line.

So far, to me it is not yet clear what is working, what is broken. I'm not on windows so can not really test.

vondele commented 4 years ago

@MichaelB7 if LTO was working fine before the patch, can you specify how your make command looked like, and can you paste/attach the build output ?

ppigazzini commented 4 years ago

@vondele I got only now the time to read the thread, sorry. LTO on windows should work only with dynamic link and copying the dlls alongside the binary, see the wiki page https://github.com/glinscott/fishtest/wiki/Building-stockfish-on-Windows#building-stockfish-with-link-time-optimization-lto

stockchess commented 4 years ago

@ppigazzini

Think wiki is outdated. I build static version with lto (see above log).

vondele commented 4 years ago

The mingw based toolchain is really fragile. I tried to cross-compile a pgo build (following the wiki procedure), which worked on ubuntu 18, but now fails on ubuntu 20 (both before and after the Makefile change).

mstembera commented 4 years ago

@ppigazzini The -flto issue has been known for a long time. Given your success I suspect the mingw toolchain got fixed at some point between gcc 8.1 and 10.1.

ppigazzini commented 4 years ago

In effect Stockfish LTO static is fine with gcc version 10.1.0 by MSYS2 and gcc version 9.2 by TDM-GCC , but still broken with gcc 8.1.0 by MinGW-w64.

MichaelB7 commented 4 years ago

this AM - all seems well - no errors stockfish.zip

Makefile.zip

@vondele 

MichaelB7@VM-894787 MINGW64 ~/home/Github/Stockfish/src (master)
$ gcc -v
Using built-in specs.
COLLECT_GCC=C:\msys64\mingw64\bin\gcc.exe
COLLECT_LTO_WRAPPER=C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.1.0/lto-wrapper.exe
Target: x86_64-w64-mingw32
Configured with: ../gcc-10.1.0/configure --prefix=/mingw64 --with-local-prefix=/mingw64/local --build=x86_64-w64-mingw32 --host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --with-native-system-header-dir=/mingw64/x86_64-w64-mingw32/include --libexecdir=/mingw64/lib --enable-bootstrap --with-arch=x86-64 --with-tune=generic --enable-languages=c,lto,c++,fortran,ada,objc,obj-c++ --enable-shared --enable-static --enable-libatomic --enable-threads=posix --enable-graphite --enable-fully-dynamic-string --enable-libstdcxx-filesystem-ts=yes --enable-libstdcxx-time=yes --disable-libstdcxx-pch --disable-libstdcxx-debug --disable-isl-version-check --enable-lto --enable-libgomp --disable-multilib --enable-checking=release --disable-rpath --disable-win32-registry --disable-nls --disable-werror --disable-symvers --disable-plugin --with-libiconv --with-system-zlib --with-gmp=/mingw64 --with-mpfr=/mingw64 --with-mpc=/mingw64 --with-isl=/mingw64 --with-pkgversion='Rev3, Built by MSYS2 project' --with-bugurl=https://sourceforge.net/projects/msys2 --with-gnu-as --with-gnu-ld
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 10.1.0 (Rev3, Built by MSYS2 project)

MichaelB7@VM-894787 MINGW64 ~/home/Github/Stockfish/src (master)
$ mke  (note: alias mke='make -j profile-build ARCH=x86-64-modern COMP=mingw'

Config:
debug: 'no'
sanitize: 'no'
optimize: 'yes'
arch: 'x86_64'
bits: '64'
kernel: 'MINGW64_NT-10.0-18363'
os: 'Windows_NT'
prefetch: 'yes'
popcnt: 'yes'
sse: 'yes'
pext: 'no'

Flags:
CXX: g++
CXXFLAGS: -Wall -Wcast-qual -fno-exceptions -std=c++11  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto
LDFLAGS:  -static -Wall -Wcast-qual -fno-exceptions -std=c++11  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto

Testing config sanity. If this fails, try 'make help' ...

Step 1/4. Building instrumented executable ...
C:/Program Files/Git/mingw64/bin/make.exe ARCH=x86-64-modern COMP=mingw gcc-profile-make
make[1]: Entering directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'
C:/Program Files/Git/mingw64/bin/make.exe ARCH=x86-64-modern COMP=mingw \
EXTRACXXFLAGS='-fprofile-generate' \
EXTRALDFLAGS='-lgcov' \
all
make[2]: Entering directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o benchmark.o benchmark.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o bitbase.o bitbase.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o bitboard.o bitboard.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o endgame.o endgame.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o evaluate.o evaluate.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o main.o main.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o material.o material.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o misc.o misc.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o movegen.o movegen.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o movepick.o movepick.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o pawns.o pawns.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o position.o position.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o psqt.o psqt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o search.o search.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o thread.o thread.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o timeman.o timeman.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o tt.o tt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o uci.o uci.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o ucioption.o ucioption.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o tune.o tune.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o tbprobe.o syzygy/tbprobe.cpp
g++ -o stockfish.exe benchmark.o bitbase.o bitboard.o endgame.o evaluate.o main.o material.o misc.o movegen.o movepick.o pawns.o position.o psqt.o search.o thread.o timeman.o tt.o uci.o ucioption.o tune.o tbprobe.o -lgcov -static -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto
make[2]: Leaving directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'
make[1]: Leaving directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'

Step 2/4. Running benchmark for pgo-build ...
./stockfish.exe bench > /dev/null

Position: 1/47

Position: 2/47

Position: 3/47

Position: 4/47

Position: 5/47

Position: 6/47

Position: 7/47

Position: 8/47

Position: 9/47

Position: 10/47

Position: 11/47

Position: 12/47

Position: 13/47

Position: 14/47

Position: 15/47

Position: 16/47

Position: 17/47

Position: 18/47

Position: 19/47

Position: 20/47

Position: 21/47

Position: 22/47

Position: 23/47

Position: 24/47

Position: 25/47

Position: 26/47

Position: 27/47

Position: 28/47

Position: 29/47

Position: 30/47

Position: 31/47

Position: 32/47

Position: 33/47

Position: 34/47

Position: 35/47

Position: 36/47

Position: 37/47

Position: 38/47

Position: 39/47

Position: 40/47

Position: 41/47

Position: 42/47

Position: 43/47

Position: 44/47

Position: 45/47

Position: 46/47

Position: 47/47

===========================
Total time (ms) : 4142
Nodes searched  : 4789930
Nodes/second    : 1156429

Step 3/4. Building optimized executable ...
C:/Program Files/Git/mingw64/bin/make.exe ARCH=x86-64-modern COMP=mingw objclean
make[1]: Entering directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'
make[1]: Leaving directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'
C:/Program Files/Git/mingw64/bin/make.exe ARCH=x86-64-modern COMP=mingw gcc-profile-use
make[1]: Entering directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'
C:/Program Files/Git/mingw64/bin/make.exe ARCH=x86-64-modern COMP=mingw \
EXTRACXXFLAGS='-fprofile-use -fno-peel-loops -fno-tracer' \
EXTRALDFLAGS='-lgcov' \
all
make[2]: Entering directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o benchmark.o benchmark.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o bitbase.o bitbase.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o bitboard.o bitboard.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o endgame.o endgame.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o evaluate.o evaluate.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o main.o main.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o material.o material.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o misc.o misc.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o movegen.o movegen.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o movepick.o movepick.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o pawns.o pawns.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o position.o position.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o psqt.o psqt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o search.o search.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o thread.o thread.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o timeman.o timeman.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o tt.o tt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o uci.o uci.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o ucioption.o ucioption.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o tune.o tune.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o tbprobe.o syzygy/tbprobe.cpp
g++ -o stockfish.exe benchmark.o bitbase.o bitboard.o endgame.o evaluate.o main.o material.o misc.o movegen.o movepick.o pawns.o position.o psqt.o search.o thread.o timeman.o tt.o uci.o ucioption.o tune.o tbprobe.o -lgcov -static -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto
make[2]: Leaving directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'
make[1]: Leaving directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'

Step 4/4. Deleting profile data ...
C:/Program Files/Git/mingw64/bin/make.exe ARCH=x86-64-modern COMP=mingw profileclean
make[1]: Entering directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'
make[1]: Leaving directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'

MichaelB7@VM-894787 MINGW64 ~/home/Github/Stockfish/src (master)
$ stockfish bench >/dev/null

Position: 1/47

Position: 2/47

Position: 3/47

Position: 4/47

Position: 5/47

Position: 6/47

Position: 7/47

Position: 8/47

Position: 9/47

Position: 10/47

Position: 11/47

Position: 12/47

Position: 13/47

Position: 14/47

Position: 15/47

Position: 16/47

Position: 17/47

Position: 18/47

Position: 19/47

Position: 20/47

Position: 21/47

Position: 22/47

Position: 23/47

Position: 24/47

Position: 25/47

Position: 26/47

Position: 27/47

Position: 28/47

Position: 29/47

Position: 30/47

Position: 31/47

Position: 32/47

Position: 33/47

Position: 34/47

Position: 35/47

Position: 36/47

Position: 37/47

Position: 38/47

Position: 39/47

Position: 40/47

Position: 41/47

Position: 42/47

Position: 43/47

Position: 44/47

Position: 45/47

Position: 46/47

Position: 47/47

===========================
Total time (ms) : 2919
Nodes searched  : 4789930
Nodes/second    : 1640948
vondele commented 4 years ago

So, if I understand correctly, the current Makefile (lto and static) only works with the very latest gcc 10.1 ?

If so, that's a bit too recent to enforce on everybody, so I think I should revert the lto part of the patch.

unaiic commented 4 years ago

Yeah, I agree. It is the best option for now IMO.

ppigazzini commented 4 years ago

@vondele it works also with gcc 9.2 by TDM-GCC, but it's broken with gcc 8.1.0 by MinGW-w64 (and older).

vondele commented 4 years ago

@ppigazzini but TDM-GCC is not default either, right. So for the makefile the following comment would be right

# To use LTO and static linking on windows, the mingw tools chain requires at least gcc version 10.1 (or TDM-GCC version 9.2), and is know to fail with gcc 8.1.0.

or can you formulate it more precisely.

FauziAkram commented 4 years ago

I have absolutely zero experience in these things, however, I just checked. My current gcc version is 4.9.2 If it's beneficial to update it to the latest 10.1 just inform me how to do that.

ppigazzini commented 4 years ago

I have absolutely zero experience in these things, however, I just checked. My current gcc version is 4.9.2 If it's beneficial to update it to the latest 10.1 just inform me how to do that.

Simply install msys2, read the wiki.

Build Stockfish https://github.com/glinscott/fishtest/wiki/Building-stockfish-on-Windows

Run the worker on Windows https://github.com/glinscott/fishtest/wiki/Running-the-worker-on-Windows

vondele commented 4 years ago

@ppigazzini can you get the PGO cross compile to work on ubuntu 20.04 LTS ? I have it fail:

$ make profile-build ARCH=x86-64-modern COMP=mingw PGOBENCH="wine ./stockfish.exe bench" -j 
[...]
x86_64-w64-mingw32-c++-posix -o stockfish.exe benchmark.o bitbase.o bitboard.o endgame.o evaluate.o main.o material.o misc.o movegen.o movepick.o pawns.o position.o psqt.o search.o thread.o timeman.o tt.o uci.o ucioption.o tune.o tbprobe.o -lgcov -static -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto
/usr/bin/x86_64-w64-mingw32-ld: /tmp/stockfish.exe.6g0fFK.ltrans0.ltrans.o:<artificial>:(.text+0x302): undefined reference to `__gcov_indirect_call_profiler_v3'

This is new on ubuntu 20.04 for me.

ppigazzini commented 4 years ago

@ppigazzini but TDM-GCC is not default either, right. So for the makefile the following comment would be right

# To use LTO and static linking on windows, the mingw tools chain requires at least gcc version 10.1 (or TDM-GCC version 9.2), and is know to fail with gcc 8.1.0.

or can you formulate it more precisely.

@vondele there is not a MinGW-w64 default on Windows. I personally like the msys2 project because is actively developed and has many packages (even QT static to build cutechess-cli), so during the years I rewrote the wiki pages in order to use msys2 as first choice. TDM-GCC is an alternative project (run by one person) that at the moment provide gcc 9.2

ppigazzini commented 4 years ago

@ppigazzini can you get the PGO cross compile to work on ubuntu 20.04 LTS ? I have it fail:

$ make profile-build ARCH=x86-64-modern COMP=mingw PGOBENCH="wine ./stockfish.exe bench" -j 
[...]
x86_64-w64-mingw32-c++-posix -o stockfish.exe benchmark.o bitbase.o bitboard.o endgame.o evaluate.o main.o material.o misc.o movegen.o movepick.o pawns.o position.o psqt.o search.o thread.o timeman.o tt.o uci.o ucioption.o tune.o tbprobe.o -lgcov -static -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto
/usr/bin/x86_64-w64-mingw32-ld: /tmp/stockfish.exe.6g0fFK.ltrans0.ltrans.o:<artificial>:(.text+0x302): undefined reference to `__gcov_indirect_call_profiler_v3'

This is new on ubuntu 20.04 for me.

I'll try tonight.

vondele commented 4 years ago

I've made the pull request for the revert, maybe somebody can test this works as expected on Windows.

mstembera commented 4 years ago

@vondele I confirm the Makefile from the new PR now works for me. Thanks!

ppigazzini commented 4 years ago

@vondele on Ubuntu 20.04 (MinGW-w64 gcc) PGO fails, same problem some years ago (the slow windows Abrok compiles ...). Anyway on Ubuntu 18.04 MinGW-w64 gcc 7.3 is able to make a static build with PGO and LTO. On Windows MinGW-w64 gcc >= 9.x is finally able to make static build with LTO (PGO always worked fine).

mstembera commented 4 years ago

FYI it's possible to check the gcc/mingw version in the makefile. https://stackoverflow.com/questions/5188267/checking-the-gcc-version-in-a-makefile

firetea commented 4 years ago

@vondele I have the same issue as you (on Ubuntu 20.04) . Any idea on how to solve this? :)

MichaelB7 commented 4 years ago

@firetea install gcc 10

firetea commented 4 years ago

@MichaelB7 Thanks, but I try to compile using mingw-w64 and I think the latest version there is is 9.3. If I have misunderstood something then please correct me :)

vondele commented 4 years ago

@firetea no I haven't been able to solve this, this was filed as a bug in ubuntu: https://bugs.launchpad.net/ubuntu/+source/gcc-mingw-w64/+bug/1883933 but it hasn't seen activity in the last month or so. It might be possible to install the older version on mingw (based on 7.3), but I haven't tried this.

firetea commented 4 years ago

@vondele Thanks, I tried it using 7.3 and it worked.

vondele commented 4 years ago

@firetea, do you mind sharing how you installed mingw 7.3 ?

ppigazzini commented 4 years ago

@vondele consider to use LXD/LXC, it's already installed on Ubuntu 20.04

sudo lxd init
# accept all defaults
lxc launch ubuntu:18.04 u1804
lxc exec u1804 -- bash
# do things in container, then exit
# lxc file pull u1804/root/somestuff.tgz somestuff.tgz
lxc stop u1804
lxc delete u1804
firetea commented 4 years ago

@vondele I considered editing /etc/apt/sources.list to add "bionic main" or similar and do an apt-get update, but in the end I went for the dirty solution to download the deb-packages from https://packages.ubuntu.com/bionic/devel/gcc-mingw-w64, i.e I downloaded gcc-mingw-w64-base_7.3.0-11ubuntu1+20.2build1_amd64.deb
g++-mingw-w64-x86-64_7.3.0-11ubuntu1+20.2build1_amd64.deb gcc-mingw-w64-x86-64_7.3.0-11ubuntu1+20.2build1_amd64.deb
libisl19_0.19-1_amd64.deb (everything clickable and linked). Then I installed them using dpkg -i I considered using the "update-alternatives" to switch which compiler would be used, but I didn't need to (because link group x86_64-w64-mingw32-gcc was missing). And then I just rebuilt. So no dependency hell :)

vondele commented 4 years ago

@ppigazzini that works... I've added the 'stuff in the container' explicitly below.

# accept all defaults
sudo lxd init
sudo lxc launch ubuntu:18.04 u1804
sudo lxc exec u1804 -- bash

apt-get update
apt-get install build-essential g++-mingw-w64 wine-stable
git clone https://github.com/official-stockfish/Stockfish.git
cd Stockfish/src/
make profile-build ARCH=x86-64-modern COMP=mingw PGOBENCH="wine ./stockfish.exe bench" -j
exit

sudo lxc file pull u1804/root/Stockfish/src/stockfish.exe ./stockfish.exe
sudo lxc stop u1804
sudo lxc delete u1804