Possible miss specification of search paths

[pjshelton]

Hi, I downloaded the following file from your website. winlibs-x86_64-posix-seh-gcc-11.2.0-llvm-13.0.0-mingw-w64ucrt-9.0.0-r2.7z

If I do the following:

echo // >dummy
g++.exe -std=c++2a -x c++ -v -E - <dummy

it returns the following

Using built-in specs.
COLLECT_GCC=g++.exe
OFFLOAD_TARGET_NAMES=nvptx-none
Target: x86_64-w64-mingw32
Configured with: ../configure --prefix=/R/winlibs64_stage/inst_gcc-11.2.0/share/gcc --build=x86_64-w64-mingw32 --host=x86_64-w64-mingw32 --enable-offload-targets=nvptx-none --with-pkgversion='MinGW-W64 x86_64-ucrt-posix-seh, built by Brecht Sanders' --with-tune=generic --enable-checking=release --enable-threads=posix --disable-sjlj-exceptions --disable-libunwind-exceptions --disable-serial-configure --disable-bootstrap --enable-host-shared --enable-plugin --disable-default-ssp --disable-rpath --enable-libstdcxx-pch --enable-libstdcxx-time=yes --disable-libstdcxx-debug --disable-version-specific-runtime-libs --with-stabs --disable-symvers --enable-languages=c,c++,fortran,lto,objc,obj-c++,d,jit --disable-gold --disable-nls --disable-stage1-checking --disable-win32-registry --disable-multilib --enable-ld --enable-libquadmath --enable-libada --enable-libssp --enable-libstdcxx --enable-lto --enable-fully-dynamic-string --enable-libgomp --enable-graphite --enable-mingw-wildcard --with-mpc=/d/Prog/winlibs64_stage/custombuilt --with-mpfr=/d/Prog/winlibs64_stage/custombuilt --with-gmp=/d/Prog/winlibs64_stage/custombuilt --with-isl=/d/Prog/winlibs64_stage/custombuilt --enable-install-libiberty --enable-__cxa_atexit --without-included-gettext --with-diagnostics-color=auto --enable-clocale=generic --with-libiconv --with-system-zlib --with-build-sysroot=/R/winlibs64_stage/gcc-11.2.0/build_mingw/mingw-w64 CFLAGS=-I/d/Prog/winlibs64_stage/custombuilt/include/libdl-win32
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 11.2.0 (MinGW-W64 x86_64-ucrt-posix-seh, built by Brecht Sanders)
COLLECT_GCC_OPTIONS='-std=c++20' '-v' '-E' '-shared-libgcc' '-mtune=generic' '-march=x86-64'
 c:/compilers/mingw64/bin/../libexec/gcc/x86_64-w64-mingw32/11.2.0/cc1plus.exe -E -quiet -v -iprefix c:\compilers\mingw64\bin\../lib/gcc/x86_64-w64-mingw32/11.2.0/ -D_REENTRANT - -mtune=generic -march=x86-64 -std=c++20 -dumpbase -
ignoring duplicate directory "c:/compilers/mingw64/lib/gcc/../../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../../include/c++/11.2.0"
ignoring duplicate directory "c:/compilers/mingw64/lib/gcc/../../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../../include/c++/11.2.0/x86_64-w64-mingw32"
ignoring duplicate directory "c:/compilers/mingw64/lib/gcc/../../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../../include/c++/11.2.0/backward"
ignoring duplicate directory "c:/compilers/mingw64/lib/gcc/../../lib/gcc/x86_64-w64-mingw32/11.2.0/include"
ignoring nonexistent directory "R:/winlibs64_stage/inst_gcc-11.2.0/share/gcc/include"
ignoring nonexistent directory "/R/winlibs64_stage/inst_gcc-11.2.0/share/gcc/include"
ignoring duplicate directory "c:/compilers/mingw64/lib/gcc/../../lib/gcc/x86_64-w64-mingw32/11.2.0/include-fixed"
ignoring duplicate directory "c:/compilers/mingw64/lib/gcc/../../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../../x86_64-w64-mingw32/include"
ignoring nonexistent directory "/mingw/include"
#include "..." search starts here:
#include <...> search starts here:
 c:\compilers\mingw64\bin\../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../../include/c++/11.2.0
 c:\compilers\mingw64\bin\../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../../include/c++/11.2.0/x86_64-w64-mingw32
 c:\compilers\mingw64\bin\../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../../include/c++/11.2.0/backward
 c:\compilers\mingw64\bin\../lib/gcc/x86_64-w64-mingw32/11.2.0/include
 c:\compilers\mingw64\bin\../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../../include
 c:\compilers\mingw64\bin\../lib/gcc/x86_64-w64-mingw32/11.2.0/include-fixed
 c:\compilers\mingw64\bin\../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../../x86_64-w64-mingw32/include
End of search list.
cc1plus.exe: warning:  is shorter than expected
# 0 "<stdin>"
# 0 "<built-in>"
# 0 "<command-line>"
# 1 "<stdin>"
COMPILER_PATH=c:/compilers/mingw64/bin/../libexec/gcc/x86_64-w64-mingw32/11.2.0/;c:/compilers/mingw64/bin/../libexec/gcc/;c:/compilers/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../../x86_64-w64-mingw32/bin/
LIBRARY_PATH=c:/compilers/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/11.2.0/;c:/compilers/mingw64/bin/../lib/gcc/;c:/compilers/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../../x86_64-w64-mingw32/lib/../lib/;c:/compilers/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../../lib/;c:/compilers/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../../x86_64-w64-mingw32/lib/;c:/compilers/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../
COLLECT_GCC_OPTIONS='-std=c++20' '-v' '-E' '-shared-libgcc' '-mtune=generic' '-march=x86-64'

I draw your attention to the following part.

ignoring duplicate directory "c:/compilers/mingw64/lib/gcc/../../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../../include/c++/11.2.0"
ignoring duplicate directory "c:/compilers/mingw64/lib/gcc/../../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../../include/c++/11.2.0/x86_64-w64-mingw32"
ignoring duplicate directory "c:/compilers/mingw64/lib/gcc/../../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../../include/c++/11.2.0/backward"
ignoring duplicate directory "c:/compilers/mingw64/lib/gcc/../../lib/gcc/x86_64-w64-mingw32/11.2.0/include"
ignoring nonexistent directory "R:/winlibs64_stage/inst_gcc-11.2.0/share/gcc/include"
ignoring nonexistent directory "/R/winlibs64_stage/inst_gcc-11.2.0/share/gcc/include"
ignoring duplicate directory "c:/compilers/mingw64/lib/gcc/../../lib/gcc/x86_64-w64-mingw32/11.2.0/include-fixed"
ignoring duplicate directory "c:/compilers/mingw64/lib/gcc/../../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../../x86_64-w64-mingw32/include"
ignoring nonexistent directory "/mingw/include"

in particular, the ignoring nonexistent directory ... lines.

Your setup is leaving directories that do not exist on my system in the specification of the compiler search paths. In particular, the R:/winlibs64_stage/inst_gcc-11.2.0/share/gcc/include and /mingw31/include.

This has caused the building of the build2 and odb toolchains (https://build2.org/, and https://www.codesynthesis.com/products/odb/) to fail.

Could you please have a look at your build recipe and see if you can cleanup the stored search paths so that all of them have mingw64 as their root.

[brechtsanders]

The idea of the winlibs build is to be relocatable. That's why gcc was configured with the --disable-rpath (do not hardcode runtime library paths) flag. So it looks like some paths are still hardcoded in gcc.

But paths that are ignored shouldn't affect the build process.

Why do you conclude that this is why you can't build build2 and odb?

When I try to build them:

• build2 fails with: error: 'unique_ptr' was not declared in this scope
• odb with: error: 'gcc_checking_assert' was not declared in this scope

[pjshelton]

This conclusion is from two weeks of discussion with one of the codesynthesis coders. First, that build2 error is a missing header inclusion. And they know about it. It has been fixed in the staged release. The build2 bootstrap builds it's self three times before it is finished. and the versions after the first are build by the previous just completed compilation.

And the second compilation uses build2 to build it's self. But it quickly terminates with

terminate called after throwing an instance of 'butl::invalid_basic_path<char>'
  what():  invalid filesystem path

Which, after a couple of days of to-ing and from-ing with the coder from codesynthesis, came down to the the fact that build2 does something with paths in the gcc search path. The nonexistent directories are causing build2 to crash. So last night, they updated the staged version of build2 to treat the nonexistent directories the same way as gcc does.

But yea, if there was anyway to catch those last couple of paths and "fix" them that would be great.

[brechtsanders]

Glad to hear you found a solution.

If you want that fixed you should report this as a bug to the gcc developers. Clearly even with --disable-rpath the path is still hardcoded here and there.

I have built several thousands of projects from source (see: https://github.com/brechtsanders/winlibs_recipes) and it's the first time this issue pops up though. But maybe that's because build2 is itself also a build toolchain.

I'm just curious: are you trying to combine build2 with the winlibs toolchain?

[pjshelton]

The project I am working on needs c++ object permanence. I am in the process of checking out odb, however I either run ./configure (which can not be done using the vanilla command line) or use its build2 recipe. That means that I have to also build build2. The parts of odb that link to my project need to be built using the same compiler as the rest of the project.

I am actually using cmake to create makefile build recipes for both Borland v5.4 and MinGW builds. But odb does not have a cmake recipe.

So while I am not actually trying to combine build2 with the winlibs toolchain, I am trying to compile build2 and, more importantly, odb with the winlibs mingw64 compiler.

[brechtsanders]

I'm not familiar with build2 with the winlibs but it looks to me that it's a whole ecosystem with build system, build toolchain and package manager all in one.

Personally I would go for configure in MSYS2 shell.

In fact I just tried that, but it seems like odb 2.4.0 (which seems like the latest stable version) has a lot of C++ build errors on recent g++.

And the more recent odb development version 2.5.0-b.21 doesn't have configure it seems.

[brechtsanders]

Patching build2 0.14.0 to ignore non-existing directorys should fix it. Something like this:

patch -ulbf libbutl/libbutl/process.cxx << EOF
@@ -3,2 +3,3 @@

+#include <memory>
 #include <libbutl/process.hxx>
EOF
patch -ulbf build2/libbuild2/cc/gcc.cxx << EOF
@@ -3,2 +3,3 @@

+#include <sys/stat.h>
 #include <libbuild2/scope.hxx>
@@ -336,6 +337,11 @@
       {
-        dir_path d (l, b, (e != string::npos ? e - b : e));
+        struct stat fileinfo;
+        string entry (l, b, (e != string::npos ? e - b : e));
+        // Skip this entry if it is not an existing directory
+        if (stat(entry.c_str(), &fileinfo) == 0 && S_ISDIR(fileinfo.st_mode)) {
+          dir_path d (entry);

-        if (find (r.begin (), r.end (), d.normalize ()) == r.end ())
-          r.emplace_back (move (d));
+          if (find (r.begin (), r.end (), d.normalize ()) == r.end ())
+            r.emplace_back (move (d));
+        }

EOF

(sorry for using C-style stat(), but somehow std::filesystem::exists() wouldn't work)

Could you try that?

[pjshelton]

I am humbled and honoured that you would make this effort for me.

I will get to trying your patch in the next couple of days.

[pjshelton]

I have applied your patch and am rerunning the build script. it completes the first pass without trouble, and creates a b-boot.exe that does not terminate with the invalid filesystem error. However it does seem to take a very long time to compile the project. it is still processing the second pass some 12 hours after I started it.

[brechtsanders]

I have totally no experience with build2. You should ask whoever made build2 why it takes so much longer. I'm already glad I could get you from not building at all to a working (but slow) build.