bfgroup / b2

B2 makes it easy to build C++ projects, everywhere.
https://www.bfgroup.xyz/b2/
Boost Software License 1.0
76 stars 228 forks source link

Boost build don't seem to work with GCC on darwin arm64 #152

Closed gracicot closed 1 year ago

gracicot commented 2 years ago

Make sure you completed the following tasks

Environment and version details

notice: found boost-build.jam at /Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/boost-build.jam
notice: loading B2 from /Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/kernel/bootstrap.jam
notice: Searching '/etc' '/Users/myself' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/kernel' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/kernel' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/util' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/build' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/tools' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/contrib' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/.' for site-config configuration file 'site-config.jam'.
notice: Configuration file 'site-config.jam' not found in '/etc' '/Users/myself' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/kernel' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/kernel' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/util' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/build' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/tools' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/contrib' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/.'.
notice: Searching '/Users/myself' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/kernel' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/kernel' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/util' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/build' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/tools' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/contrib' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/.' for user-config configuration file 'user-config.jam'.
notice: Configuration file 'user-config.jam' not found in '/Users/myself' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/kernel' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/kernel' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/util' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/build' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/tools' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/contrib' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/.'.
notice: [cmdline-cfg] toolset gcc not previously configured; attempting to auto-configure now
notice: will use 'g++' for gcc, condition <toolset>gcc-11
notice: using gcc libraries :: <toolset>gcc-11 :: /nix/store/a74idnqpcb9wr9p30fza69wzvcir72pf-gcc-wrapper-11.2.0/bin /nix/store/a74idnqpcb9wr9p30fza69wzvcir72pf-gcc-wrapper-11.2.0/lib /nix/store/a74idnqpcb9wr9p30fza69wzvcir72pf-gcc-wrapper-11.2.0/lib32 /nix/store/a74idnqpcb9wr9p30fza69wzvcir72pf-gcc-wrapper-11.2.0/lib64
notice: using gcc archiver :: <toolset>gcc-11 :: ar
warning: toolset gcc initialization: can not find tool windres
warning: initialized from
notice: using rc compiler :: <toolset>gcc-11 :: /nix/store/a74idnqpcb9wr9p30fza69wzvcir72pf-gcc-wrapper-11.2.0/bin/as
...found 1 target...

Brief problem description

Any boost build commands ends with lol_add failed due to reached limit of 19 elements, even the debug commands

Steps to reproduce the issue

Create those files in a directory: vcpkg.json

{
    "name": "test-b2",
    "version-string": "0.1",
    "dependencies": [
        "boost-system"
    ]
}

CMakeLists.txt

cmake_minimum_required(VERSION 3.21)
project(testb2 CXX)
add_executable(e main.cpp)

main.cpp

#include <cstdio>
int main() { std::puts("hello gcc"); }

vcpkg.nix

{
  stdenv,
  fetchFromGitHub,
  gcc,
  ninja,
  cmake,
  lib,
  git,
  cacert,
  runtimeShell
}:

stdenv.mkDerivation rec {
  pname = "vcpkg";
  version = "74ff2a04415691361b07b7dbd1d4762ad79bd82c";

  src = fetchFromGitHub {
    owner = "microsoft";
    repo = "vcpkg-tool";
    rev = "2022-03-30";
    sha256 = "06l0zpqlh20dc300zlf47xjpk5x2db390pdir77jyk3cfpx7i3x2";
  };

  vcpkg_src = fetchFromGitHub {
    owner = "microsoft";
    repo = "vcpkg";
    rev = "74ff2a04415691361b07b7dbd1d4762ad79bd82c";
    sha256 = "0w32hz2055gjj33p2yi7h5sqn3ccv5c5wa6m9k7nxb56g3l05cjs";
  };

  buildInputs = [cmake ninja git cacert];

  vcpkgScript = ''
    #!${runtimeShell}
    vcpkg_hash=$(echo -n "${placeholder "out"}" | sha256sum | cut -f1 -d ' ')
    vcpkg_root_path="$HOME/.vcpkg/roots/$vcpkg_hash"
    if [[ ! -d "$vcpkg_root_path" ]]; then
      mkdir -p "$vcpkg_root_path"
      touch "$vcpkg_root_path/.vcpkg-root"
      ln -s ${placeholder "out"}/share/vcpkg/{docs,ports,scripts,triplets,versions} "$vcpkg_root_path/"
      ln -s ${placeholder "out"}/bin/vcpkg "$vcpkg_root_path/"
    fi
    VCPKG_DOWNLOADS="$vcpkg_root_path/downloads" VCPKG_ROOT="$vcpkg_root_path" ${placeholder "out"}/share/vcpkg/vcpkg "$@"
  '';

  passAsFile = [ "vcpkgScript" ];
  installPhase = ''
    cmake --build . --target=install --config=Release
    mkdir -p $out/share/vcpkg
    cp --preserve=mode -r ${vcpkg_src}/{docs,ports,scripts,triplets,.vcpkg-root,versions} $out/share/vcpkg
    mv $out/bin/vcpkg $out/share/vcpkg/vcpkg
    cp $vcpkgScriptPath $out/bin/vcpkg
    chmod +x $out/bin/vcpkg
    touch $out/share/vcpkg/vcpkg.disable-metrics
  '';
  enableParallelBuilding = true;
}

shell.nix

{
    pkgs   ? import <nixpkgs> {},
    stdenv ? pkgs.gcc11Stdenv
}:
rec {
  testb2 = stdenv.mkDerivation rec {
    name = "testb2";
    version = "dev-0.1";
    vcpkg = pkgs.callPackage ./vcpkg.nix {};
    buildInputs = with pkgs; [
      gcc11
      ninja
      cmake
      git
      perl
      gdb
      vcpkg

      pkg-config-unwrapped

      # vcpkg dependencies
      curl
      zip
      unzip
      cacert
    ];

    hardeningDisable = [ "stackprotector" ];

    VCPKG_ROOT = "~/.vcpkg/roots/${builtins.hashString "sha256" "${vcpkg}"}";
    shellHook = ''
      vcpkg > /dev/null 2>&1
    '';
  };
}

Run those commands in the terminal where the files were created:

nix-shell
mkdir build && cd build
cmake .. -DCMAKE_TOOLCHAIN_FILE=${VCPKG_ROOT}/scripts/buildsystems/vcpkg.cmake -DCMAKE_CXX_COMPILER=g++ -GNinja

Problem also reported at microsoft/vcpkg#24659

Boost build was compiled with -fno-stack-protector since it would not compile with stack protection.

Actual behavior summary

The output of any b2 command will output the following:

notice: found boost-build.jam at /Users/software/test2/build/vcpkg_installed/arm64-osx/tools/boost-build/boost-build.jam
notice: loading B2 from /Users/software/test2/build/vcpkg_installed/arm64-osx/tools/boost-build/src/kernel/bootstrap.jam
lol_add failed due to reached limit of 19 elements

If a project of yours is blocked due to this bug, please, mention it explicitly.

Our development may continue on Linux virtual machines instead

Expected behavior summary

The error should not happen and b2 should work as well as if it was compiled using clang.

gracicot commented 2 years ago

Everything seems to work normally when building boost build with clang

grafikrobot commented 2 years ago

Which gcc build are you using. I.e. how did you install gcc? And what version?

gracicot commented 2 years ago

I'm using the gcc11 package from nix. If you use those nix files, you'll have the exact same environment as me and it should reproduce the issue on a M1 computer.

grafikrobot commented 2 years ago

I don't use nix. Or have an Apple M1 machine available to me. How/where does Nix obtain the gcc build?

gracicot commented 2 years ago

I'm pretty sure they build gcc themselves since they only publish reproducible builds. I'm gonna try to reproduce the error on x86 using cross compiling, but I'm not sure it's gonna run without using a arm64 virtual machine.

gracicot commented 2 years ago

I got more info about the version of boost build by compiling it using clang instead b2 -v

B2 Version 4.7. OS=MACOSX.
  Copyright 1993-2002 Christopher Seiwald and Perforce Software, Inc.
  Copyright 2001 David Turner.
  Copyright 2001-2004 David Abrahams.
  Copyright 2002-2019 Rene Rivera.
  Copyright 2003-2015 Vladimir Prus.

  DEFAULTS: jobs = 8

b2 --debug-configuration toolset=gcc

notice: found boost-build.jam at /Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/boost-build.jam
notice: loading B2 from /Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/kernel/bootstrap.jam
notice: Searching '/etc' '/Users/myself' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/kernel' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/kernel' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/util' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/build' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/tools' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/contrib' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/.' for site-config configuration file 'site-config.jam'.
notice: Configuration file 'site-config.jam' not found in '/etc' '/Users/myself' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/kernel' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/kernel' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/util' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/build' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/tools' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/contrib' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/.'.
notice: Searching '/Users/myself' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/kernel' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/kernel' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/util' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/build' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/tools' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/contrib' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/.' for user-config configuration file 'user-config.jam'.
notice: Configuration file 'user-config.jam' not found in '/Users/myself' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/kernel' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/kernel' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/util' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/build' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/tools' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/contrib' '/Users/myself/project/build/vcpkg_installed/arm64-osx/tools/boost-build/src/.'.
notice: [cmdline-cfg] toolset gcc not previously configured; attempting to auto-configure now
notice: will use 'g++' for gcc, condition <toolset>gcc-11
notice: using gcc libraries :: <toolset>gcc-11 :: /nix/store/a74idnqpcb9wr9p30fza69wzvcir72pf-gcc-wrapper-11.2.0/bin /nix/store/a74idnqpcb9wr9p30fza69wzvcir72pf-gcc-wrapper-11.2.0/lib /nix/store/a74idnqpcb9wr9p30fza69wzvcir72pf-gcc-wrapper-11.2.0/lib32 /nix/store/a74idnqpcb9wr9p30fza69wzvcir72pf-gcc-wrapper-11.2.0/lib64
notice: using gcc archiver :: <toolset>gcc-11 :: ar
warning: toolset gcc initialization: can not find tool windres
warning: initialized from
notice: using rc compiler :: <toolset>gcc-11 :: /nix/store/a74idnqpcb9wr9p30fza69wzvcir72pf-gcc-wrapper-11.2.0/bin/as
...found 1 target...
gracicot commented 2 years ago

Would it help if I'm creating a guide to reproduce the issue that don't use nix? I created those nix files simply to make the issue easier to reproduce.

grafikrobot commented 2 years ago

Would it help if I'm creating a guide to reproduce the issue that don't use nix? I created those nix files simply to make the issue easier to reproduce.

Yes it would :-)

gracicot commented 2 years ago

Hmm, I tried to reproduce the problem but it seems that there's another issue blocking me from getting the same result. Everything seems to work on a X86 processor, but not on ARM. Here's the steps:

The expected results:

B2 4.9-git

The actual result:

Programs hang with no output. Running ./b2 install --prefix=./install hangs too.

From what I've read from the readme is that only clang is actually supported on MacOS. Is it the case?

grafikrobot commented 2 years ago

From what I've read from the readme is that only clang is actually supported on MacOS. Is it the case?

Currently yes, as that's the one configuration I can easily test. But the gcc support should work on almost all platforms. Assuming that gcc on the platform produced usable executable with the minimal flags we pass to it (I've run into some that don't). And to some extent it works on macOS also since it works for x86.

Questions...

gracicot commented 2 years ago

Oh! It seems that b2 -v outputs something:

B2 Version 4.9. OS=MACOSX.
  Copyright 1993-2002 Christopher Seiwald and Perforce Software, Inc.
  Copyright 2001 David Turner.
  Copyright 2001-2004 David Abrahams.
  Copyright 2002-2019 Rene Rivera.
  Copyright 2003-2015 Vladimir Prus.

  DEFAULTS: jobs = 8

I'm running b2 --version in the source directory, the bootstrap script created the executable at the root of the project folder.

If it helps, here's the compiler command from the bootstrap script:

g++ -DNDEBUG builtins.cpp class.cpp command.cpp compile.cpp constants.cpp cwd.cpp debug.cpp debugger.cpp execcmd.cpp execnt.cpp execunix.cpp filesys.cpp filent.cpp fileunix.cpp frames.cpp function.cpp glob.cpp hash.cpp hcache.cpp hdrmacro.cpp headers.cpp jam_strings.cpp jam.cpp jamgram.cpp lists.cpp make.cpp make1.cpp md5.cpp mem.cpp modules.cpp native.cpp object.cpp option.cpp output.cpp parse.cpp pathnt.cpp pathsys.cpp pathunix.cpp regexp.cpp rules.cpp scan.cpp search.cpp startup.cpp subst.cpp sysinfo.cpp timestamp.cpp variable.cpp w32_getreg.cpp modules/order.cpp modules/path.cpp modules/property-set.cpp modules/regex.cpp modules/sequence.cpp modules/set.cpp -o b2
grafikrobot commented 2 years ago

g++ -DNDEBUG builtins.cpp class.cpp command.cpp compile.cpp constants.cpp cwd.cpp debug.cpp debugger.cpp execcmd.cpp execnt.cpp execunix.cpp filesys.cpp filent.cpp fileunix.cpp frames.cpp function.cpp glob.cpp hash.cpp hcache.cpp hdrmacro.cpp headers.cpp jam_strings.cpp jam.cpp jamgram.cpp lists.cpp make.cpp make1.cpp md5.cpp mem.cpp modules.cpp native.cpp object.cpp option.cpp output.cpp parse.cpp pathnt.cpp pathsys.cpp pathunix.cpp regexp.cpp rules.cpp scan.cpp search.cpp startup.cpp subst.cpp sysinfo.cpp timestamp.cpp variable.cpp w32_getreg.cpp modules/order.cpp modules/path.cpp modules/property-set.cpp modules/regex.cpp modules/sequence.cpp modules/set.cpp -o b2

Hm.. Interesting.. There should be a few more options on that command. Thanks for the info. I'll investigate at least why some of the expected options appear missing.

stknecht commented 2 years ago

Hi, by any chance any progress on this issue. I am having the same problem with my M1 macbook and gcc... Thanks! Stefan

Sayani26 commented 2 years ago

Hi! Any progress on this? I am encountering the same error while running b2. The bootstrapping is done but prints this error trying to run b2. Using gcc11.2 macOS Monterey.

gracicot commented 2 years ago

@Sayani26 My temporary fix was to use Clang. Here's how to do this properly so developer won't notice the fix is there.

To do that with vcpkg I kept stdenv as my nix base environment so CMake would find Clang by default.

Then, I set in the environment using my nix script so it takes this value:

VCPKG_DEFAULT_TRIPLET = "${stdenv.system}-gcc";

Where ${stdenv.system} is x86_64-darwin or aarch64-linux, something like that. The goal of this is that when coupled with direnv, we automatically gain those environment variables as soon as we enter the directory.

So after that I created 4 triplet file for all permutation of linux, darwin, x86_64, aarch64. They are all copy paste of the default one except aarch64-darwin and x86_64-darwin:

set(VCPKG_TARGET_ARCHITECTURE arm64) # keep the right architecture here
set(VCPKG_CRT_LINKAGE dynamic)
set(VCPKG_LIBRARY_LINKAGE static)

set(VCPKG_CMAKE_SYSTEM_NAME Darwin)
set(VCPKG_OSX_ARCHITECTURES arm64)

set(VCPKG_CHAINLOAD_TOOLCHAIN_FILE "${CMAKE_CURRENT_LIST_DIR}/../toolchains/osx-gcc.cmake")

In this file I kept all the same thing as the xyz-osx triplet file except I set a chainload toolchain file at the end. I created this toolchain to be the same as the default one shipped by vcpkg, but modified so it uses gcc as the compiler:

if(NOT _ARMREST_OSX_TOOLCHAIN)
set(_ARMREST_OSX_TOOLCHAIN 1)
if(CMAKE_HOST_SYSTEM_NAME STREQUAL "Darwin")
    set(CMAKE_CROSSCOMPILING OFF CACHE BOOL "")
    set(CMAKE_SYSTEM_VERSION "${CMAKE_HOST_SYSTEM_VERSION}" CACHE STRING "")
else()
    set(CMAKE_SYSTEM_VERSION "17.0.0" CACHE STRING "")
endif()
set(CMAKE_SYSTEM_NAME Darwin CACHE STRING "")

set(CMAKE_MACOSX_RPATH ON CACHE BOOL "")

if(NOT DEFINED CMAKE_SYSTEM_PROCESSOR)
    if(VCPKG_TARGET_ARCHITECTURE STREQUAL "x64")
       set(CMAKE_SYSTEM_PROCESSOR x86_64 CACHE STRING "")
    elseif(VCPKG_TARGET_ARCHITECTURE STREQUAL "x86")
       set(CMAKE_SYSTEM_PROCESSOR x86 CACHE STRING "")
    elseif(VCPKG_TARGET_ARCHITECTURE STREQUAL "arm64")
       set(CMAKE_SYSTEM_PROCESSOR arm64 CACHE STRING "")
    else()
       set(CMAKE_SYSTEM_PROCESSOR "${CMAKE_HOST_SYSTEM_PROCESSOR}" CACHE STRING "")
    endif()
endif()

# HERE!!
set(CMAKE_CXX_COMPILER "g++")
set(CMAKE_C_COMPILER "gcc")

get_property( _CMAKE_IN_TRY_COMPILE GLOBAL PROPERTY IN_TRY_COMPILE )
if(NOT _CMAKE_IN_TRY_COMPILE)
    string(APPEND CMAKE_C_FLAGS_INIT " -fPIC ${VCPKG_C_FLAGS} ")
    string(APPEND CMAKE_CXX_FLAGS_INIT " -fPIC ${VCPKG_CXX_FLAGS} ")
    string(APPEND CMAKE_C_FLAGS_DEBUG_INIT " ${VCPKG_C_FLAGS_DEBUG} ")
    string(APPEND CMAKE_CXX_FLAGS_DEBUG_INIT " ${VCPKG_CXX_FLAGS_DEBUG} ")
    string(APPEND CMAKE_C_FLAGS_RELEASE_INIT " ${VCPKG_C_FLAGS_RELEASE} ")
    string(APPEND CMAKE_CXX_FLAGS_RELEASE_INIT " ${VCPKG_CXX_FLAGS_RELEASE} ")

    string(APPEND CMAKE_SHARED_LINKER_FLAGS_INIT " ${VCPKG_LINKER_FLAGS} ")
    string(APPEND CMAKE_EXE_LINKER_FLAGS_INIT " ${VCPKG_LINKER_FLAGS} ")
    string(APPEND CMAKE_SHARED_LINKER_FLAGS_DEBUG_INIT " ${VCPKG_LINKER_FLAGS_DEBUG} ")
    string(APPEND CMAKE_EXE_LINKER_FLAGS_DEBUG_INIT " ${VCPKG_LINKER_FLAGS_DEBUG} ")
    string(APPEND CMAKE_SHARED_LINKER_FLAGS_RELEASE_INIT " ${VCPKG_LINKER_FLAGS_RELEASE} ")
    string(APPEND CMAKE_EXE_LINKER_FLAGS_RELEASE_INIT " ${VCPKG_LINKER_FLAGS_RELEASE} ")
endif()
endif()

So now that all of that is done, you can configure your CMake profile to set the right variables:

"VCPKG_TARGET_TRIPLET": "$env{VCPKG_DEFAULT_TRIPLET}",
"VCPKG_OVERLAY_TRIPLETS": "${sourceDir}/triplets",
"CMAKE_TOOLCHAIN_FILE": "$env{VCPKG_ROOT}/scripts/buildsystems/vcpkg.cmake",
"CMAKE_C_COMPILER": "gcc",
"CMAKE_CXX_COMPILER": "g++",

Remember, VCPKG_DEFAULT_TRIPLET is the environment variable we set in the beginning.

Also, we explicitly set gcc and g++ for our own project.

What this does is that on darwin only, we instruct vcpkg to use a chainload toolchain file. So in a sense, vcpkg will think we are kind of cross compiling but for the current platform, with the exception that we use GCC as the default compiler as instructed in the toolchain file.

Now that all of this is setup, we can tell vcpkg that boost-build is a host dependency, a tool used by the host machine when cross compiling. In this case, vcpkg will use VCPKG_HOST_TRIPLET, which we let it be the default. So for boost-build only, vcpkg will use the arm64-osx triplet, which uses the default compiler, which uses Clang.

It's done like so:

{
    "name": "your project",
    "version-string": "0.1",
    "dependencies":[
        {
            "name": "boost-build",
            "host": true
        },
        "boost-asio",
        "boost-beast",
        "boost-system",
        // ...
}

Done! Transparently vcpkg will use Clang but only for boost-build, which won't create linking problem further down the road.

grafikrobot commented 1 year ago

Minor update.. I now have minimal access to an M1 machine so I can better test this. I realized also why the extra compile options are missing. When using the --cxx=.. option if you don't also pass in the toolset to use we assume that you are going to specify options in the --cxxflags=... But if you specify the toolset, ie --cxx=g++ gcc the full compile shows up. For example:

gcc104 (homebrew):bfg-b2 grafik$ ./src/engine/build.sh --cxx=g++-12 gcc

###

###

### Using 'gcc' toolset.

###

###

g++-12 (Homebrew GCC 12.2.0) 12.2.0

Copyright (C) 2022 Free Software Foundation, Inc.

This is free software; see the source for copying conditions.  There is NO

warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

###

###

> g++-12 -x c++ -std=c++11 -O2 -s -DNDEBUG builtins.cpp class.cpp command.cpp compile.cpp constants.cpp cwd.cpp debug.cpp debugger.cpp execcmd.cpp execnt.cpp execunix.cpp filesys.cpp filent.cpp fileunix.cpp frames.cpp function.cpp glob.cpp hash.cpp hcache.cpp hdrmacro.cpp headers.cpp jam_strings.cpp jam.cpp jamgram.cpp lists.cpp make.cpp make1.cpp md5.cpp mem.cpp modules.cpp native.cpp object.cpp option.cpp output.cpp parse.cpp pathnt.cpp pathsys.cpp pathunix.cpp regexp.cpp rules.cpp scan.cpp search.cpp startup.cpp subst.cpp sysinfo.cpp timestamp.cpp variable.cpp w32_getreg.cpp modules/order.cpp modules/path.cpp modules/property-set.cpp modules/regex.cpp modules/sequence.cpp modules/set.cpp -o b2

ld: warning: option -s is obsolete and being ignored
grafikrobot commented 1 year ago

Good news is that I can easily replicate the problem, even with the corrected compile options. Bad news is that I can't debug in the M1 machine I have access to (permission limitations).

grafikrobot commented 1 year ago

Even worse news.. I tried the in-progress b2-5.0.0 version. And that causes the Mac linker to crash when invoked by gcc. So I suspect the homebrew gcc has some serious bugs in the M1 mac.

biergaizi commented 1 year ago

I'm now experiencing the same problem on Gentoo Prefix. Gentoo Prefix is an environment that provides basically a Gentoo Linux userland on a foreign system like macOS and it uses the GNU toolchain (but not glibc) by default. This is similar to how nixOS works.

I really need to test my application on GCC, as its behavior is possibly GCC-specific (auto-vectorization). Gentoo Prefix allows me to build everything with GCC without worrying about incompatibilities, as all shared-library dependencies would be built via GCC as well.

Unfortunately Boost cannot be built due to this b2 problem. When b2 is built with GCC 12.1.0 and GCC 12.2.0, running b2 crashes with Segmentation fault: 11. Running it inside lldb cannot replicate the crash - it gets stuck in a loop instead.

I noticed that Iain Sandoe's GCC source had just fixed a broken Thread-Local Storage problem - https://github.com/iains/gcc-12-branch/issues/9. I'm not sure if these problems are related... The commit date is Sep 25, 2022. Homebrew did not incorporate this at that time fix due to another technical problem, see this issue for discussion: https://github.com/Homebrew/homebrew-core/issues/110673 (Iain is GCC's Darwin target maintainer, his downstream GCC source contains experimental fixes and patches, and they're currently used by Apple M1 systems, including Homebrew).

ec2-user@ip-10-0-4-85 ~/empty $ b2
Segmentation fault: 11

ec2-user@ip-10-0-4-85 ~/empty $ lldb b2
(lldb) target create "b2"
Current executable set to 'b2' (arm64).
(lldb) r
Process 24726 launched: '/Users/ec2-user/gentoo/usr/bin/b2' (arm64)
Process 24726 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
    frame #0: 0x000000010001c128 b2`list_copy(LIST*) + 56
b2`list_copy:
->  0x10001c128 <+56>: add    w1, w1, #0x1
    0x10001c12c <+60>: lsl    w3, w0, w1
    0x10001c130 <+64>: cmp    w19, w3
    0x10001c134 <+68>: b.gt   0x10001c128               ; <+56>
Target 0: (b2) stopped.
(lldb) q
Quitting LLDB will kill one or more processes. Do you really want to proceed: [Y/n] y
ec2-user@ip-10-0-4-85 ~/empty $ 

GCC version:

ec2-user@ip-10-0-4-85 ~/empty $ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/Users/ec2-user/gentoo/usr/libexec/gcc/arm64-apple-darwin22/12.2.0/lto-wrapper
Target: arm64-apple-darwin22
Configured with: /Users/ec2-user/gentoo/var/tmp/portage/sys-devel/gcc-12.2.0/work/gcc-12-branch-gcc-12.2-darwin-r0/configure --host=arm64-apple-darwin22 --build=arm64-apple-darwin22 --prefix=/Users/ec2-user/gentoo/usr --bindir=/Users/ec2-user/gentoo/usr/arm64-apple-darwin22/gcc-bin/12.2.0 --includedir=/Users/ec2-user/gentoo/usr/lib/gcc/arm64-apple-darwin22/12.2.0/include --datadir=/Users/ec2-user/gentoo/usr/share/gcc-data/arm64-apple-darwin22/12.2.0 --mandir=/Users/ec2-user/gentoo/usr/share/gcc-data/arm64-apple-darwin22/12.2.0/man --infodir=/Users/ec2-user/gentoo/usr/share/gcc-data/arm64-apple-darwin22/12.2.0/info --with-gxx-include-dir=/Users/ec2-user/gentoo/usr/lib/gcc/arm64-apple-darwin22/12.2.0/include/g++-v12 --with-python-dir=/share/gcc-data/arm64-apple-darwin22/12.2.0/python --enable-languages=c,c++,objc,obj-c++,fortran --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --disable-libunwind-exceptions --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion='Gentoo 12.2.0 p1' --disable-esp --enable-libstdcxx-time --disable-libstdcxx-pch --enable-shared --enable-threads=posix --with-local-prefix=/Users/ec2-user/gentoo/usr --with-native-system-header-dir=/Users/ec2-user/gentoo/MacOSX.sdk/usr/include --disable-multilib --disable-fixed-point --enable-version-specific-runtime-libs --enable-libgomp --disable-libssp --disable-libada --disable-cet --disable-systemtap --disable-valgrind-annotations --disable-vtable-verify --disable-libvtv --without-zstd --enable-lto --without-isl --disable-libsanitizer --enable-default-pie --disable-default-ssp
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 12.2.0 (Gentoo 12.2.0 p1) 
biergaizi commented 1 year ago

I confirm that the Thread-Local Storage bug in GCC is unrelated. It must be something else...

biergaizi commented 1 year ago

Building with optimization level -Og allows one to replicate the original problem. Even better, now with memory corruptions too.

ec2-user@ip-10-0-4-85 ~/empty $ /Users/ec2-user/b2-4.9.3/src/engine/b2
notice: loading B2 from /Users/ec2-user/b2-4.9.3/src/kernel/bootstrap.jam
lol_add failed due to reached limit of 19 elements
b2(4009,0x205fe8100) malloc: *** error for object 0x16bd424d0: pointer being freed was not allocated
b2(4009,0x205fe8100) malloc: *** set a breakpoint in malloc_error_break to debug
Abort trap: 6

Furthermore, enabling assertion by removing -DNDEBUG allows us to catch the problem earlier.

ec2-user@ip-10-0-4-85 ~/empty $ lldb /Users/ec2-user/b2-4.9.3/src/engine/b2                                                    [77/1904]
(lldb) target create "/Users/ec2-user/b2-4.9.3/src/engine/b2"                                                                           
Current executable set to '/Users/ec2-user/b2-4.9.3/src/engine/b2' (arm64).                                                             
(lldb) r                                                                                                                                
Process 4312 launched: '/Users/ec2-user/b2-4.9.3/src/engine/b2' (arm64)                                                                 
notice: loading B2 from /Users/ec2-user/b2-4.9.3/src/kernel/bootstrap.jam                                                               
lol_add failed due to reached limit of 19 elements                                                                                      
Assertion failed: (object_get_item( obj )->header.magic == OBJECT_MAGIC), function object_validate, file object.cpp, line 288.          
Process 4312 stopped                                                                                                                    
* thread #1, queue = 'com.apple.main-thread', stop reason = hit program assert                                                          
    frame #4: 0x0000000100020d7c b2`::object_validate(obj=<unavailable>) at object.cpp:288:24                                           
   285  static void object_validate( OBJECT * obj )                                                                                     
   286  {                                                                                                                               
   287      assert( obj );                                                                                                              
-> 288      assert( object_get_item( obj )->header.magic == OBJECT_MAGIC );                                                             
   289  }                                                                                                                               
   290                                                                                                                                  
   291                                                                                                                                  
Target 0: (b2) stopped.

When all optimizations are disabled -O0, the behavior is the same - it's either a crash when running outside the debugger, or an infinite loop in the function get_bucket() when running inside the debugger. The argument size=1876891088 of get_bucket() is interesting, it looks extremely large. Does it suggest some kind of uninitialized variable, integer overflow or memory corruption issues?

Run directly:

ec2-user@ip-10-0-4-85 ~/empty $ /Users/ec2-user/b2-4.9.3/src/engine/b2                                                                  
notice: loading B2 from /Users/ec2-user/b2-4.9.3/src/kernel/bootstrap.jam                                                               
Segmentation fault: 11

Run in lldb:

(lldb) r
Process 3574 launched: '/Users/ec2-user/b2-4.9.3/src/engine/b2' (arm64)
notice: loading B2 from /Users/ec2-user/b2-4.9.3/src/kernel/bootstrap.jam
Process 3574 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
    frame #0: 0x000000010002a560 b2`::get_bucket(size=1876891088) at lists.cpp:22:18
   19   static int32_t get_bucket( int32_t size )
   20   {
   21       int32_t bucket = 0;
-> 22       while ( size > ( int32_t(1) << bucket ) ) ++bucket;
   23       return bucket;
   24   }
   25  
Target 0: (b2) stopped.
(lldb) mn
error: 'mn' is not a valid command.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
  * frame #0: 0x000000010002a560 b2`::get_bucket(size=1876891088) at lists.cpp:22:18
    frame #1: 0x000000010002a58c b2`::list_alloc(size=1876891088) at lists.cpp:28:32
    frame #2: 0x000000010002a924 b2`list_copy(l=0x000000016fdf10f0) at lists.cpp:137:24
    frame #3: 0x0000000100014688 b2`frame_get_local(frame=0x000000016fdf0f30, idx=2) at function.cpp:452:21
    frame #4: 0x000000010001f350 b2`function_run(function_=0x0000600002c08580, frame=0x000000016fdf0f30) at function.cpp:4344:20
    frame #5: 0x000000010000785c b2`evaluate_rule(rule=0x0000000100b59d58, rulename=0x00000001018282d8, frame=0x000000016fdf0f30) at compile.cpp:150:30
    frame #6: 0x00000001000079cc b2`call_rule(rulename=0x00000001018282d8, caller_frame=0x000000016fdf17e0) at compile.cpp:195:27
    frame #7: 0x00000001000418b8 b2`property_set_create(frame=0x000000016fdf17e0, flags=0) at property-set.cpp:164:31
    frame #8: 0x000000010001eda8 b2`function_run(function_=0x0000600000c04ba0, frame=0x000000016fdf17e0) at function.cpp:4299:25
    frame #9: 0x000000010000785c b2`evaluate_rule(rule=0x0000000100b59d88, rulename=0x000000010181a1b0, frame=0x000000016fdf17e0) at compile.cpp:150:30
    frame #10: 0x0000000100014e18 b2`::function_call_rule(function=0x0000600002c19400, frame=0x000000016fdf1fc0, s=0x0000000100061300, n_args=0, unexpanded="create", file=0x00000001018a9278, line=575) at function.cpp:642:27
    frame #11: 0x00000001000208e0 b2`function_run(function_=0x0000600002c19400, frame=0x000000016fdf1fc0) at function.cpp:5004:47
    frame #12: 0x000000010000785c b2`evaluate_rule(rule=0x0000000100b59e48, rulename=0x000000010182be98, frame=0x000000016fdf1fc0) at compile.cpp:150:30
    frame #13: 0x0000000100014e18 b2`::function_call_rule(function=0x0000600002c1d080, frame=0x000000016fdf2cf0, s=0x0000000100061300, n_args=0, unexpanded="property-set.empty", file=0x000000010185bf38, line=639) at function.cpp:642:27
    frame #14: 0x00000001000208e0 b2`function_run(function_=0x0000600002c1d080, frame=0x000000016fdf2cf0) at function.cpp:5004:47
    frame #15: 0x00000001000352e8 b2`::parse_impl(frame=0x000000016fdf2cf0) at parse.cpp:72:18
    frame #16: 0x0000000100035364 b2`parse_file(f=0x000000010185bf38, frame=0x000000016fdf2cf0) at parse.cpp:84:15
    frame #17: 0x000000010002119c b2`function_run(function_=0x0000600002c04c80, frame=0x000000016fdf2cf0) at function.cpp:5249:27
    frame #18: 0x000000010000785c b2`evaluate_rule(rule=0x0000000100b0b698, rulename=0x000000010181db88, frame=0x000000016fdf2cf0) at compile.cpp:150:30
    frame #19: 0x0000000100014e18 b2`::function_call_rule(function=0x0000600002c04c00, frame=0x000000016fdf34d0, s=0x0000000100061300, n_args=3, unexpanded="load", file=0x000000010181d3e8, line=294) at function.cpp:642:27
    frame #20: 0x00000001000208e0 b2`function_run(function_=0x0000600002c04c00, frame=0x000000016fdf34d0) at function.cpp:5004:47
    frame #21: 0x000000010000785c b2`evaluate_rule(rule=0x0000000101815988, rulename=0x000000010181ad80, frame=0x000000016fdf34d0) at compile.cpp:150:30
    frame #22: 0x0000000100014e18 b2`::function_call_rule(function=0x0000600002c1aa80, frame=0x000000016fdf4200, s=0x0000000100061300, n_args=1, unexpanded="import", file=0x00000001018ac6a8, line=66) at function.cpp:642:27
    frame #23: 0x00000001000208e0 b2`function_run(function_=0x0000600002c1aa80, frame=0x000000016fdf4200) at function.cpp:5004:47
    frame #24: 0x00000001000352e8 b2`::parse_impl(frame=0x000000016fdf4200) at parse.cpp:72:18
    frame #25: 0x0000000100035364 b2`parse_file(f=0x00000001018ac6a8, frame=0x000000016fdf4200) at parse.cpp:84:15
    frame #26: 0x000000010002119c b2`function_run(function_=0x0000600002c04c80, frame=0x000000016fdf4200) at function.cpp:5249:27
    frame #27: 0x000000010000785c b2`evaluate_rule(rule=0x0000000100b0b698, rulename=0x000000010181db88, frame=0x000000016fdf4200) at compile.cpp:150:30
    frame #28: 0x0000000100014e18 b2`::function_call_rule(function=0x0000600002c04c00, frame=0x000000016fdf49e0, s=0x0000000100061300, n_args=3, unexpanded="load", file=0x000000010181d3e8, line=294) at function.cpp:642:27
    frame #29: 0x00000001000208e0 b2`function_run(function_=0x0000600002c04c00, frame=0x000000016fdf49e0) at function.cpp:5004:47
    frame #30: 0x000000010000785c b2`evaluate_rule(rule=0x0000000101815988, rulename=0x000000010181ad80, frame=0x000000016fdf49e0) at compile.cpp:150:30
    frame #31: 0x0000000100014e18 b2`::function_call_rule(function=0x0000600002c18180, frame=0x000000016fdf5710, s=0x0000000100061300, n_args=1, unexpanded="import", file=0x00000001018a52d0, line=30) at function.cpp:642:27
    frame #32: 0x00000001000208e0 b2`function_run(function_=0x0000600002c18180, frame=0x000000016fdf5710) at function.cpp:5004:47
    frame #33: 0x00000001000352e8 b2`::parse_impl(frame=0x000000016fdf5710) at parse.cpp:72:18
    frame #34: 0x0000000100035364 b2`parse_file(f=0x00000001018a52d0, frame=0x000000016fdf5710) at parse.cpp:84:15
    frame #35: 0x000000010002119c b2`function_run(function_=0x0000600002c04c80, frame=0x000000016fdf5710) at function.cpp:5249:27
    frame #36: 0x000000010000785c b2`evaluate_rule(rule=0x0000000100b0b698, rulename=0x000000010181db88, frame=0x000000016fdf5710) at compile.cpp:150:30
    frame #37: 0x0000000100014e18 b2`::function_call_rule(function=0x0000600002c04c00, frame=0x000000016fdf5ef0, s=0x0000000100061300, n_args=3, unexpanded="load", file=0x000000010181d3e8, line=294) at function.cpp:642:27
    frame #38: 0x00000001000208e0 b2`function_run(function_=0x0000600002c04c00, frame=0x000000016fdf5ef0) at function.cpp:5004:47
    frame #39: 0x000000010000785c b2`evaluate_rule(rule=0x0000000101815988, rulename=0x000000010181ad80, frame=0x000000016fdf5ef0) at compile.cpp:150:30
    frame #40: 0x0000000100014e18 b2`::function_call_rule(function=0x0000600002c17b00, frame=0x000000016fdf6c20, s=0x0000000100061300, n_args=1, unexpanded="import", file=0x000000010189cbf0, line=16) at function.cpp:642:27
    frame #41: 0x00000001000208e0 b2`function_run(function_=0x0000600002c17b00, frame=0x000000016fdf6c20) at function.cpp:5004:47
    frame #42: 0x00000001000352e8 b2`::parse_impl(frame=0x000000016fdf6c20) at parse.cpp:72:18
    frame #43: 0x0000000100035364 b2`parse_file(f=0x000000010189cbf0, frame=0x000000016fdf6c20) at parse.cpp:84:15
    frame #44: 0x000000010002119c b2`function_run(function_=0x0000600002c04c80, frame=0x000000016fdf6c20) at function.cpp:5249:27
    frame #45: 0x000000010000785c b2`evaluate_rule(rule=0x0000000100b0b698, rulename=0x000000010181db88, frame=0x000000016fdf6c20) at compile.cpp:150:30
    frame #46: 0x0000000100014e18 b2`::function_call_rule(function=0x0000600002c04c00, frame=0x000000016fdf7400, s=0x0000000100061300, n_args=3, unexpanded="load", file=0x000000010181d3e8, line=294) at function.cpp:642:27
    frame #47: 0x00000001000208e0 b2`function_run(function_=0x0000600002c04c00, frame=0x000000016fdf7400) at function.cpp:5004:47
    frame #48: 0x000000010000785c b2`evaluate_rule(rule=0x0000000101815988, rulename=0x000000010181ad80, frame=0x000000016fdf7400) at compile.cpp:150:30
    frame #49: 0x0000000100014e18 b2`::function_call_rule(function=0x0000600002c15f00, frame=0x000000016fdf8130, s=0x0000000100061300, n_args=1, unexpanded="import", file=0x0000000101895738, line=40) at function.cpp:642:27
    frame #50: 0x00000001000208e0 b2`function_run(function_=0x0000600002c15f00, frame=0x000000016fdf8130) at function.cpp:5004:47
    frame #51: 0x00000001000352e8 b2`::parse_impl(frame=0x000000016fdf8130) at parse.cpp:72:18
    frame #52: 0x0000000100035364 b2`parse_file(f=0x0000000101895738, frame=0x000000016fdf8130) at parse.cpp:84:15
    frame #53: 0x000000010002119c b2`function_run(function_=0x0000600002c04c80, frame=0x000000016fdf8130) at function.cpp:5249:27
    frame #54: 0x000000010000785c b2`evaluate_rule(rule=0x0000000100b0b698, rulename=0x000000010181db88, frame=0x000000016fdf8130) at compile.cpp:150:30
    frame #55: 0x0000000100014e18 b2`::function_call_rule(function=0x0000600002c04c00, frame=0x000000016fdf8910, s=0x0000000100061300, n_args=3, unexpanded="load", file=0x000000010181d3e8, line=294) at function.cpp:642:27
    frame #56: 0x00000001000208e0 b2`function_run(function_=0x0000600002c04c00, frame=0x000000016fdf8910) at function.cpp:5004:47
    frame #57: 0x000000010000785c b2`evaluate_rule(rule=0x0000000101815988, rulename=0x000000010181ad80, frame=0x000000016fdf8910) at compile.cpp:150:30
    frame #58: 0x0000000100014e18 b2`::function_call_rule(function=0x0000600002c15a00, frame=0x000000016fdf9640, s=0x0000000100061300, n_args=1, unexpanded="import", file=0x000000010188d248, line=28) at function.cpp:642:27
    frame #59: 0x00000001000208e0 b2`function_run(function_=0x0000600002c15a00, frame=0x000000016fdf9640) at function.cpp:5004:47
    frame #60: 0x00000001000352e8 b2`::parse_impl(frame=0x000000016fdf9640) at parse.cpp:72:18
    frame #61: 0x0000000100035364 b2`parse_file(f=0x000000010188d248, frame=0x000000016fdf9640) at parse.cpp:84:15
    frame #62: 0x000000010002119c b2`function_run(function_=0x0000600002c04c80, frame=0x000000016fdf9640) at function.cpp:5249:27
    frame #63: 0x000000010000785c b2`evaluate_rule(rule=0x0000000100b0b698, rulename=0x000000010181db88, frame=0x000000016fdf9640) at compile.cpp:150:30
    frame #64: 0x0000000100014e18 b2`::function_call_rule(function=0x0000600002c04c00, frame=0x000000016fdf9e20, s=0x0000000100061300, n_args=3, unexpanded="load", file=0x000000010181d3e8, line=294) at function.cpp:642:27
    frame #65: 0x00000001000208e0 b2`function_run(function_=0x0000600002c04c00, frame=0x000000016fdf9e20) at function.cpp:5004:47
    frame #66: 0x000000010000785c b2`evaluate_rule(rule=0x0000000101815988, rulename=0x000000010181ad80, frame=0x000000016fdf9e20) at compile.cpp:150:30
    frame #67: 0x0000000100014e18 b2`::function_call_rule(function=0x0000600002c15880, frame=0x000000016fdfab50, s=0x0000000100061300, n_args=1, unexpanded="import", file=0x000000010188cb28, line=12) at function.cpp:642:27
    frame #68: 0x00000001000208e0 b2`function_run(function_=0x0000600002c15880, frame=0x000000016fdfab50) at function.cpp:5004:47
    frame #69: 0x00000001000352e8 b2`::parse_impl(frame=0x000000016fdfab50) at parse.cpp:72:18
    frame #70: 0x0000000100035364 b2`parse_file(f=0x000000010188cb28, frame=0x000000016fdfab50) at parse.cpp:84:15
    frame #71: 0x000000010002119c b2`function_run(function_=0x0000600002c04c80, frame=0x000000016fdfab50) at function.cpp:5249:27
    frame #72: 0x000000010000785c b2`evaluate_rule(rule=0x0000000100b0b698, rulename=0x000000010181db88, frame=0x000000016fdfab50) at compile.cpp:150:30
    frame #73: 0x0000000100014e18 b2`::function_call_rule(function=0x0000600002c04c00, frame=0x000000016fdfb330, s=0x0000000100061300, n_args=3, unexpanded="load", file=0x000000010181d3e8, line=294) at function.cpp:642:27
    frame #74: 0x00000001000208e0 b2`function_run(function_=0x0000600002c04c00, frame=0x000000016fdfb330) at function.cpp:5004:47
    frame #75: 0x000000010000785c b2`evaluate_rule(rule=0x0000000101815988, rulename=0x000000010181ad80, frame=0x000000016fdfb330) at compile.cpp:150:30
    frame #76: 0x0000000100014e18 b2`::function_call_rule(function=0x0000600002c04500, frame=0x000000016fdfc060, s=0x0000000100061300, n_args=1, unexpanded="import", file=0x0000000101828218, line=12) at function.cpp:642:27
    frame #77: 0x00000001000208e0 b2`function_run(function_=0x0000600002c04500, frame=0x000000016fdfc060) at function.cpp:5004:47
    frame #78: 0x00000001000352e8 b2`::parse_impl(frame=0x000000016fdfc060) at parse.cpp:72:18
    frame #79: 0x0000000100035364 b2`parse_file(f=0x0000000101828218, frame=0x000000016fdfc060) at parse.cpp:84:15
    frame #80: 0x000000010002119c b2`function_run(function_=0x0000600002c04c80, frame=0x000000016fdfc060) at function.cpp:5249:27
    frame #81: 0x000000010000785c b2`evaluate_rule(rule=0x0000000100b0b698, rulename=0x000000010181db88, frame=0x000000016fdfc060) at compile.cpp:150:30
    frame #82: 0x0000000100014e18 b2`::function_call_rule(function=0x0000600002c04c00, frame=0x000000016fdfc840, s=0x0000000100061300, n_args=3, unexpanded="load", file=0x000000010181d3e8, line=294) at function.cpp:642:27
    frame #83: 0x00000001000208e0 b2`function_run(function_=0x0000600002c04c00, frame=0x000000016fdfc840) at function.cpp:5004:47
    frame #84: 0x000000010000785c b2`evaluate_rule(rule=0x0000000101815988, rulename=0x000000010181ad80, frame=0x000000016fdfc840) at compile.cpp:150:30
    frame #85: 0x0000000100014e18 b2`::function_call_rule(function=0x0000600002c04080, frame=0x000000016fdfd5e8, s=0x0000000100061300, n_args=1, unexpanded="import", file=0x000000010181a6e0, line=135) at function.cpp:642:27
    frame #86: 0x00000001000208e0 b2`function_run(function_=0x0000600002c04080, frame=0x000000016fdfd5e8) at function.cpp:5004:47
    frame #87: 0x00000001000352e8 b2`::parse_impl(frame=0x000000016fdfd5e8) at parse.cpp:72:18
    frame #88: 0x0000000100035364 b2`parse_file(f=0x000000010181a6e0, frame=0x000000016fdfd5e8) at parse.cpp:84:15
    frame #89: 0x000000010003f104 b2`b2::startup::bootstrap(frame=0x000000016fdfd5e8) at startup.cpp:275:15
    frame #90: 0x0000000100027894 b2`guarded_main(argc=0, argv=0x000000016fdfefd0) at jam.cpp:577:48
    frame #91: 0x0000000100027ab0 b2`main(argc=1, argv=0x000000016fdfefc8) at jam.cpp:629:30
    frame #92: 0x00000001aa733e50 dyld`start + 2544
(lldb) q
Quitting LLDB will kill one or more processes. Do you really want to proceed: [Y/n] y
ec2-user@ip-10-0-4-85 ~/empty $ lldb /Users/ec2-user/b2-4.9.3/src/engine/b2                                                        
(lldb) target create "/Users/ec2-user/b2-4.9.3/src/engine/b2"
Current executable set to '/Users/ec2-user/b2-4.9.3/src/engine/b2' (arm64).
(lldb) r
Process 3583 launched: '/Users/ec2-user/b2-4.9.3/src/engine/b2' (arm64)
notice: loading B2 from /Users/ec2-user/b2-4.9.3/src/kernel/bootstrap.jam
Process 3583 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
    frame #0: 0x000000010002a560 b2`::get_bucket(size=1876891088) at lists.cpp:22:18
   19   static int32_t get_bucket( int32_t size )
   20   {
   21       int32_t bucket = 0;
-> 22       while ( size > ( int32_t(1) << bucket ) ) ++bucket;
   23       return bucket;
   24   }
   25  
Target 0: (b2) stopped.
biergaizi commented 1 year ago

I suspect it's some kind of memory corruption, unfortunately I cannot debug it further without external help from tools. GCC's Address Sanitizer currently doesn't support Apple M1 (on macOS at least, it probably works fine on Linux), valgrind also doesn't support recent macOS at all, even on x86.

biergaizi commented 1 year ago

It's just unbelievable, we got a Heisenbug here. I've found the problem could be made to appear and disappear in a totally unbelievable manner.

I was trying to replicate it with Homebrew instead of Gentoo prefix.

First, git checkout to branch 99742ad614537370686ab488560c1b23c6b4c95b, this is the official 4.9.3 version tag. Running ./bootstrap.sh gcc-12 creates a b4 with a size of 342991 bytes. This executable will always crash. Next, git checkout to just one commit beyond 4.9.3, which is e9d5539b127361a8eab57d0892ca175435e49354 - this is just a merge commit without real changes, but running ./bootstrap.sh gcc-12 again produces a working b4 with a size of 661151 bytes!

Now I suspect it's perhaps a complier bug that could be influenced by the filesystem. Alternatively, git history is not really linear, and perhaps the commit e9d5539b127361a8eab57d0892ca175435e49354 has something hidden?

Broken Version

bash-5.2$ ./bootstrap.sh gcc-12
Building the B2 engine..

###
###
### Using 'gcc-12' toolset.
###
###

g++-12 (Homebrew GCC 12.2.0) 12.2.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

###
###

> g++-12 -x c++ -std=c++11 -O2 -s -DNDEBUG builtins.cpp class.cpp command.cpp compile.cpp constants.cpp cwd.cpp debug.cpp debugger.cpp execcmd.cpp execnt.cpp execunix.cpp filesys.cpp filent.cpp fileunix.cpp frames.cpp function.cpp glob.cpp hash.cpp hcache.cpp hdrmacro.cpp headers.cpp jam_strings.cpp jam.cpp jamgram.cpp lists.cpp make.cpp make1.cpp md5.cpp mem.cpp modules.cpp native.cpp object.cpp option.cpp output.cpp parse.cpp pathnt.cpp pathsys.cpp pathunix.cpp regexp.cpp rules.cpp scan.cpp search.cpp startup.cpp subst.cpp sysinfo.cpp timestamp.cpp variable.cpp w32_getreg.cpp modules/order.cpp modules/path.cpp modules/property-set.cpp modules/regex.cpp modules/sequence.cpp modules/set.cpp -o b2
builtins.cpp: In function 'LIST* builtin_calc(FRAME*, int)':
builtins.cpp:530:12: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  530 |     sprintf( buffer, "%ld", result_value );
      |     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from jam.h:348,
                 from builtins.cpp:7:
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
builtins.cpp: In function 'LIST* builtin_backtrace(FRAME*, int)':
builtins.cpp:1381:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
 1381 |         sprintf( buf, "%d", line );
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
builtins.cpp: In function 'LIST* builtin_nearest_user_location(FRAME*, int)':
builtins.cpp:1680:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
 1680 |         sprintf( buf, "%d", line );
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
builtins.cpp: In function 'LIST* builtin_md5(FRAME*, int)':
builtins.cpp:1713:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
 1713 |         sprintf( hex_output + di * 2, "%02x", digest[ di ] );
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
builtins.cpp: In function 'LIST* builtin_file_open(FRAME*, int)':
builtins.cpp:1754:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
 1754 |         sprintf( buffer, "%d", fd );
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
builtins.cpp: In function 'LIST* builtin_shell(FRAME*, int)':
builtins.cpp:2120:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
 2120 |         sprintf( buffer, "%d", exit_status );
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
debugger.cpp: In function 'void debug_parent_clear(int, const char**)':
debugger.cpp:1523:12: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
 1523 |     sprintf( buf, "%d", id );
      |     ~~~~~~~^~~~~~~~~~~~~~~~~
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk/usr/include/wchar.h:90,
                 from /opt/homebrew/Cellar/gcc/12.2.0/include/c++/12/cwchar:44,
                 from /opt/homebrew/Cellar/gcc/12.2.0/include/c++/12/bits/postypes.h:40,
                 from /opt/homebrew/Cellar/gcc/12.2.0/include/c++/12/bits/char_traits.h:39,
                 from /opt/homebrew/Cellar/gcc/12.2.0/include/c++/12/string:40,
                 from object.h:16,
                 from debugger.h:13,
                 from debugger.cpp:8:
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
debugger.cpp: In function 'void debug_parent_backtrace(int, const char**)':
debugger.cpp:1581:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
 1581 |         sprintf( buf, "%d", i );
      |         ~~~~~~~^~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
debugger.cpp: In function 'void debug_mi_break_insert(int, const char**)':
debugger.cpp:1941:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
 1941 |         sprintf( buf, "%d", num_breakpoints );
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
execcmd.cpp: In function 'void argv_from_shell(const char**, LIST*, const char*, int32_t)':
execcmd.cpp:55:12: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
   55 |     sprintf( jobno, "%d", slot + 1 );
      |     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~
In file included from jam.h:348,
                 from execcmd.cpp:11:
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
filesys.cpp: In function 'void file_archivescan_impl(OBJECT*, archive_scanback, void*)':
filesys.cpp:421:20: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  421 |             sprintf( buf, "%s(%s)",
      |             ~~~~~~~^~~~~~~~~~~~~~~~
  422 |                 object_str( archive->file->name ),
      |                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  423 |                 object_str( member_file->name ) );
      |                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from jam.h:348,
                 from filesys.cpp:30:
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
fileunix.cpp: In function 'void file_archscan(const char*, scanback, void*)':
fileunix.cpp:240:20: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  240 |             sprintf( buf, "%s(%s)",
      |             ~~~~~~~^~~~~~~~~~~~~~~~
  241 |                 object_str( archive->file->name ),
      |                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  242 |                 object_str( member_file->name ) );
      |                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from jam.h:348,
                 from fileunix.cpp:30:
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
fileunix.cpp: In function 'int file_collect_archive_content_(file_archive_info_t*)':
fileunix.cpp:357:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  357 |         sprintf( buf, "%s", lar_name );
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
hcache.cpp: In function 'void hcache_done()':
hcache.cpp:397:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  397 |         sprintf( includes_count_str, "%lu", (long unsigned)list_length(
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  398 |             c->includes ) );
      |             ~~~~~~~~~~~~~~~
In file included from jam.h:348,
                 from hcache.cpp:32:
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
hcache.cpp:399:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  399 |         sprintf( hdrscan_count_str, "%lu", (long unsigned)list_length(
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  400 |             c->hdrscan ) );
      |             ~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
hcache.cpp:401:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  401 |         sprintf( time_secs_str, "%lu", (long unsigned)c->time.secs );
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
hcache.cpp:402:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  402 |         sprintf( time_nsecs_str, "%lu", (long unsigned)c->time.nsecs );
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
hcache.cpp:403:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  403 |         sprintf( age_str, "%lu", (long unsigned)c->age );
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
make.cpp: In function 'const char* target_name(TARGET*)':
make.cpp:776:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  776 |         sprintf( buf, "%s (internal node)", object_str( t->name ) );
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from jam.h:348,
                 from make.cpp:33:
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
output.cpp: In function 'OBJECT* outf_int(int)':
output.cpp:176:12: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  176 |     sprintf( buffer, "%i", value );
      |     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~
In file included from jam.h:348,
                 from output.cpp:7:
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
output.cpp: In function 'OBJECT* outf_double(double)':
output.cpp:184:12: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  184 |     sprintf( buffer, "%f", value );
      |     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
pathsys.cpp: In function 'OBJECT* path_tmpnam()':
pathsys.cpp:270:12: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  270 |     sprintf( name_buffer, "jam%lx%lx.000", pid, t );
      |     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from jam.h:348,
                 from pathsys.cpp:30:
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
scan.cpp: In function 'char* symdump(YYSYMBOL*)':
scan.cpp:716:29: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  716 |         case EOF   : sprintf( buf, "EOF"                                        ); break;
      |                      ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from jam.h:348,
                 from scan.cpp:12:
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
scan.cpp:717:29: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  717 |         case 0     : sprintf( buf, "unknown symbol %s", object_str( s->string ) ); break;
      |                      ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
scan.cpp:718:29: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  718 |         case ARG   : sprintf( buf, "argument %s"      , object_str( s->string ) ); break;
      |                      ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
scan.cpp:719:29: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  719 |         case STRING: sprintf( buf, "string \"%s\""    , object_str( s->string ) ); break;
      |                      ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
scan.cpp:720:29: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  720 |         default    : sprintf( buf, "keyword %s"       , s->keyword              ); break;
      |                      ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
timestamp.cpp: In function 'const char* timestamp_formatstr(const timestamp*, const char*)':
timestamp.cpp:185:12: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  185 |     sprintf( result2, result1, time->nsecs );
      |     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from jam.h:348,
                 from timestamp.cpp:26:
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
ld: warning: option -s is obsolete and being ignored

Building is done. To install, run:

    ./b2 install --prefix=<DIR>

bash-5.2$ ./src/engine/b2
Segmentation fault: 11
bash-5.2$ ls -l ./src/engine/b2
-rwxr-xr-x  1 homebrew  staff  342991 Feb 20 00:54 ./src/engine/b2

Good Version

bash-5.2$ git checkout e9d5539b127361a8eab57d0892ca175435e49354
Previous HEAD position was 99742ad61 Bump to 4.9.3.
HEAD is now at e9d5539b1 Merge branch 'version/4.9.3'

bash-5.2$ git clean -dxf ./
Removing b2
Removing src/engine/b2
bash-5.2$ ./bootstrap.sh gcc-12
Building the B2 engine..

###
###
### Using 'gcc-12' toolset.
###
###

g++-12 (Homebrew GCC 12.2.0) 12.2.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

###
###

> g++-12 -x c++ -std=c++11 -O2 -s -DNDEBUG bindjam.cpp builtins.cpp class.cpp command.cpp compile.cpp constants.cpp cwd.cpp debug.cpp debugger.cpp execcmd.cpp execnt.cpp execunix.cpp filent.cpp filesys.cpp fileunix.cpp frames.cpp function.cpp glob.cpp hash.cpp hcache.cpp hdrmacro.cpp headers.cpp jam_strings.cpp jam.cpp jamgram.cpp lists.cpp make.cpp make1.cpp md5.cpp mem.cpp modules.cpp native.cpp option.cpp output.cpp parse.cpp pathnt.cpp pathsys.cpp pathunix.cpp regexp.cpp rules.cpp scan.cpp search.cpp startup.cpp subst.cpp timestamp.cpp value.cpp variable.cpp w32_getreg.cpp mod_jam_builtin.cpp mod_jam_class.cpp mod_jam_errors.cpp mod_jam_modules.cpp mod_order.cpp mod_path.cpp mod_property_set.cpp mod_regex.cpp mod_sequence.cpp mod_set.cpp mod_string.cpp mod_sysinfo.cpp mod_version.cpp -o b2
builtins.cpp: In function 'LIST* builtin_calc(FRAME*, int)':
builtins.cpp:521:12: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  521 |     sprintf( buffer, "%ld", result_value );
      |     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from jam.h:348,
                 from builtins.cpp:7:
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
builtins.cpp: In function 'LIST* builtin_backtrace(FRAME*, int)':
builtins.cpp:1372:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
 1372 |         sprintf( buf, "%d", line );
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
builtins.cpp: In function 'LIST* builtin_nearest_user_location(FRAME*, int)':
builtins.cpp:1659:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
 1659 |         sprintf( buf, "%d", line );
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
builtins.cpp: In function 'LIST* builtin_md5(FRAME*, int)':
builtins.cpp:1692:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
 1692 |         sprintf( hex_output + di * 2, "%02x", digest[ di ] );
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
builtins.cpp: In function 'LIST* builtin_file_open(FRAME*, int)':
builtins.cpp:1733:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
 1733 |         sprintf( buffer, "%d", fd );
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
builtins.cpp: In function 'LIST* builtin_shell(FRAME*, int)':
builtins.cpp:2075:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
 2075 |         sprintf( buffer, "%d", exit_status );
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
debugger.cpp: In function 'void debug_parent_clear(int, const char**)':
debugger.cpp:1523:12: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
 1523 |     sprintf( buf, "%d", id );
      |     ~~~~~~~^~~~~~~~~~~~~~~~~
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk/usr/include/wchar.h:90,
                 from /opt/homebrew/Cellar/gcc/12.2.0/include/c++/12/cwchar:44,
                 from /opt/homebrew/Cellar/gcc/12.2.0/include/c++/12/bits/postypes.h:40,
                 from /opt/homebrew/Cellar/gcc/12.2.0/include/c++/12/bits/char_traits.h:39,
                 from /opt/homebrew/Cellar/gcc/12.2.0/include/c++/12/string:40,
                 from value.h:15,
                 from object.h:12,
                 from debugger.h:13,
                 from debugger.cpp:8:
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
debugger.cpp: In function 'void debug_parent_backtrace(int, const char**)':
debugger.cpp:1581:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
 1581 |         sprintf( buf, "%d", i );
      |         ~~~~~~~^~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
debugger.cpp: In function 'void debug_mi_break_insert(int, const char**)':
debugger.cpp:1941:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
 1941 |         sprintf( buf, "%d", num_breakpoints );
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
execcmd.cpp: In function 'void argv_from_shell(const char**, LIST*, const char*, int32_t)':
execcmd.cpp:55:12: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
   55 |     sprintf( jobno, "%d", slot + 1 );
      |     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~
In file included from jam.h:348,
                 from execcmd.cpp:11:
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
filesys.cpp: In function 'void file_archivescan_impl(OBJECT*, archive_scanback, void*)':
filesys.cpp:421:20: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  421 |             sprintf( buf, "%s(%s)",
      |             ~~~~~~~^~~~~~~~~~~~~~~~
  422 |                 object_str( archive->file->name ),
      |                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  423 |                 object_str( member_file->name ) );
      |                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from jam.h:348,
                 from filesys.cpp:30:
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
fileunix.cpp: In function 'void file_archscan(const char*, scanback, void*)':
fileunix.cpp:240:20: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  240 |             sprintf( buf, "%s(%s)",
      |             ~~~~~~~^~~~~~~~~~~~~~~~
  241 |                 object_str( archive->file->name ),
      |                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  242 |                 object_str( member_file->name ) );
      |                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from jam.h:348,
                 from fileunix.cpp:30:
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
fileunix.cpp: In function 'int file_collect_archive_content_(file_archive_info_t*)':
fileunix.cpp:357:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  357 |         sprintf( buf, "%s", lar_name );
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
hcache.cpp: In function 'void hcache_done()':
hcache.cpp:397:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  397 |         sprintf( includes_count_str, "%lu", (long unsigned)list_length(
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  398 |             c->includes ) );
      |             ~~~~~~~~~~~~~~~
In file included from jam.h:348,
                 from hcache.cpp:32:
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
hcache.cpp:399:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  399 |         sprintf( hdrscan_count_str, "%lu", (long unsigned)list_length(
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  400 |             c->hdrscan ) );
      |             ~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
hcache.cpp:401:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  401 |         sprintf( time_secs_str, "%lu", (long unsigned)c->time.secs );
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
hcache.cpp:402:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  402 |         sprintf( time_nsecs_str, "%lu", (long unsigned)c->time.nsecs );
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
hcache.cpp:403:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  403 |         sprintf( age_str, "%lu", (long unsigned)c->age );
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
make.cpp: In function 'const char* target_name(TARGET*)':
make.cpp:776:16: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  776 |         sprintf( buf, "%s (internal node)", object_str( t->name ) );
      |         ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from jam.h:348,
                 from make.cpp:33:
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
output.cpp: In function 'OBJECT* outf_int(int)':
output.cpp:176:12: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  176 |     sprintf( buffer, "%i", value );
      |     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~
In file included from jam.h:348,
                 from output.cpp:7:
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
output.cpp: In function 'OBJECT* outf_double(double)':
output.cpp:184:12: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  184 |     sprintf( buffer, "%f", value );
      |     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
pathsys.cpp: In function 'OBJECT* path_tmpnam()':
pathsys.cpp:270:12: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  270 |     sprintf( name_buffer, "jam%lx%lx.000", pid, t );
      |     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from jam.h:348,
                 from pathsys.cpp:30:
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
scan.cpp: In function 'char* symdump(YYSYMBOL*)':
scan.cpp:785:29: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  785 |         case EOF   : sprintf( buf, "EOF"                                        ); break;
      |                      ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from jam.h:348,
                 from scan.cpp:12:
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
scan.cpp:786:29: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  786 |         case 0     : sprintf( buf, "unknown symbol %s", object_str( s->string ) ); break;
      |                      ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
scan.cpp:787:29: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  787 |         case ARG   : sprintf( buf, "argument %s"      , object_str( s->string ) ); break;
      |                      ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
scan.cpp:788:29: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  788 |         case STRING: sprintf( buf, "string \"%s\""    , object_str( s->string ) ); break;
      |                      ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
scan.cpp:789:29: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  789 |         default    : sprintf( buf, "keyword %s"       , s->keyword              ); break;
      |                      ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
timestamp.cpp: In function 'const char* timestamp_formatstr(const timestamp*, const char*)':
timestamp.cpp:185:12: warning: 'int sprintf(char*, const char*, ...)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  185 |     sprintf( result2, result1, time->nsecs );
      |     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from jam.h:348,
                 from timestamp.cpp:26:
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/include-fixed/stdio.h:204:10: note: declared here
  204 | int      sprintf(char * __restrict, const char * __restrict, ...) __printflike(2, 3);
      |          ^~~~~~~
ld: warning: option -s is obsolete and being ignored

Building is done. To install, run:

    ./b2 install --prefix=<DIR>

bash-5.2$ ./src/engine/b2
warning: No toolsets are configured.
warning: Configuring default toolset "clang".
warning: If the default is wrong, your build may not work correctly.
warning: Use the "toolset=xxxxx" option to override our guess.
warning: For more configuration options, please consult
warning: https://www.bfgroup.xyz/b2/manual/release/index.html#bbv2.overview.configuration
...found 1 target...

bash-5.2$ ls -l ./src/engine/b2
-rwxr-xr-x  1 homebrew  staff  661151 Feb 20 00:56 ./src/engine/b2
biergaizi commented 1 year ago

As I suspected, the culprit is git's non-linear history, and it's creating much confusion and masking the actual code changes here. The merge commit https://github.com/bfgroup/b2/commit/e9d5539b127361a8eab57d0892ca175435e49354 is a non-linear merge and has multiple parents. It means it's possible that some commit that didn't appear in https://github.com/bfgroup/b2/commit/99742ad614537370686ab488560c1b23c6b4c95b could suddenly appear after the commit https://github.com/bfgroup/b2/commit/e9d5539b127361a8eab57d0892ca175435e49354 has been made, and these commits "travel back in time" after the merge.

For example, consider the commit 7eb13937ace5f8e034c956472b71365755651a3a ("Fix ODR violations") made on Sep 3, 2022. This commit shows up before both 99742ad614537370686ab488560c1b23c6b4c95b ("Bump to 4.9.3.") and e9d5539b127361a8eab57d0892ca175435e49354 ("Merge branch 'version/4.9.3"). So you may think that at both 99742ad614537370686ab488560c1b23c6b4c95b ("Bump to 4.9.3.") and e9d5539b127361a8eab57d0892ca175435e49354's point in time, they both contain the ODR fix 7eb13937ace5f8e034c956472b71365755651a3a. But this is not true. Actually, the tagged 4.9.3 version e9d5539b127361a8eab57d0892ca175435e49354 ("Bump to 4.9.3.") DOES NOT contain the ODR fix. It only appeared after the merge commit e9d5539b127361a8eab57d0892ca175435e49354 ("Merge branch 'version/4.9.3") but by traveling back in time before 7eb13937ace5f8e034c956472b71365755651a3a ("Bump to 4.9.3."). But this time travel only occurs after e9d5539b127361a8eab57d0892ca175435e49354 ("Merge branch 'version/4.9.3") has been made.

In other words, the ODR fix is actually missing from the current 4.9.3. And there are actually a huge number of changes between these two commits...

biergaizi commented 1 year ago

Progress! After running git bisect, I found the first good commit that no longer causes GCC build to crash is:

commit 60852922851c2009081de7e8406e6112a8ec17f6
Author: Rene Rivera <grafikrobot@gmail.com>
Date:   Fri Aug 5 15:29:25 2022 -0500

Port the "modules" kernel module to C++.

Lots of changes needed to port the module to C++. And various fixes to
stay ASAN clean. Also started on the "path" module to move path code
that was previously in the "modules" module.

The mention of "various fixes to stay ASAN clean" reinforces my belief that the GCC crash is caused by some kind of memory corruption. Unfortunately, these fixes are mixed into a huge commit and cannot be isolated. In the future, please keep all bug fixes strictly inside their own commits, thank you.

biergaizi commented 1 year ago

I'm able to confirm that the current git main branch also runs on Darwin ARM64 without any apparent problem. I was able to start building Boost after switching to the main branch.

Still, until the root clause of the original bug is identified, exactly what was happening is still an open question, either because b2 invokes undefined behavior or corrupts memory, or because there's a bug in GCC. The reason that the crash disappeared after 60852922851c2009081de7e8406e6112a8ec17f6 was because the bootstrap routine has been rewritten from Jam to C++, so no wonder the bug disappeared.

I'm not familiar with the code enough to investigate the actual bug, especially when both valgrind and ASAN are not available... My understanding is, b2 basically includes an interpreter / virtual machine for its Jam scripting language, and this bug occurred while the bootstrap script was being executed, so it happens deep inside the stack with many nested calls.

grafikrobot commented 1 year ago

@biergaizi thanks for the investigations. Some explanations.., The main branch contains the future 5.0.0 release. It has many, many, changes that have accumulated from a couple years worth of work. That work includes many internal changes to start porting from Jam to C++. Part of that was a lot of ASAN debugging, memory tracking, and changes to the Jam interpreter. Yes, your view of the Jam language is correct. The scripting language works by running a bytecode compiler during parsing while also running the bytecode on a stack based VM.

So, not sure what to say at this point of this bug. If it works in the future 5.0.0 version I'm not going to try and resolve it on the 4.x release branch. Because doing so would likely take serious amount of time.

biergaizi commented 1 year ago

Right now I'm debugging another problem found in flex, the old-school Unix compiler tool. I'm seeing strange behavior in a variadic function... Even when there's no more argument to process, the variadic loop between va_start() and va_end() keeps running and creates garbage in memory.

So I decided to check b2's source code as well, and... Bingo!

@@ -182,21 +179,37 @@ LIST * call_rule( OBJECT * rulename, FRAME * caller_frame, ... )
         : caller_frame->prev_user;
     inner->module = caller_frame->module;

-    va_start( va, caller_frame );
-    for ( ; ; )
+    for ( int32_t a = 0; a < args->count; ++a)
     {
-        LIST * const l = va_arg( va, LIST * );
-        if ( !l )
-            break;
-        lol_add( inner->args, l );
+        lol_add( inner->args, list_copy(lol_get(args, a)) );
     }
-    va_end( va );

Our problem is lol_add(), and it's inside a variadic loop!

Now I'm going to investigate if GCC's implementation of variadic function has problems...

biergaizi commented 1 year ago

I've found the problem.

It's not a GCC bug, GCC simply exposed an existing bug lurking inside our programs, that is, the dependence of undefined behaviors - a typical C situation.

In C programming, it's common to write a variadic function with a NULL-terminated argument list as its input format. For example, the following buggy program processes the argument "arg1", "arg2", "arg3", "arg4", then encounters the value 0 and stops.

#include <stdio.h>
#include <stdarg.h>

void f(char *x, ...)
{
    va_list ap;
    char *str;
    int args = 0;

    va_start(ap, x);
    while ((str = va_arg(ap, char *)) != NULL) {
        args++;
        printf("called %d times!\n", args);
    }

    va_end(ap);
}

int main(void)
{
    f("not included", "str1", "str2", "str3", "str4", 0);  /* WRONG! */
}

Running it on most systems will produce the following output:

called 1 times!
called 2 times!
called 3 times!
called 4 times!

However, the code is wrong and invokes undefined behavior. When you run this program on Apple M1 with GCC, you get the following output instead:

called 1 times!
called 2 times!
called 3 times!
called 4 times!
called 5 times!
called 6 times!
called 7 times!
called 8 times!

What's going on?

The function f(x, ...) only accepts arguments with pointer type char *, but we're passing the number 0, which is char (or int). Thus, we're passing an integer in place of a pointer. This is undefined behavior in C, thus, on some platforms it does not work correctly.

The fix is simply to pass NULL instead of 0.

f("not included", "str1", "str2", "str3", "str4", NULL);

The value NULL is internally defined to be (void *) 0, so it's a pointer type. A single NULL worked for me, but many say it's still not 100% safe, it's better to cast explicitly.

f("not included", "str1", "str2", "str3", "str4", (char *) NULL);
// or
f("not included", "str1", "str2", "str3", "str4", (char *) 0);

This kind of bugs is not a new problem. It's an old and classic problem in C programming. For example, in 1987, there was a popular article called The Ten Commandments for C Programmers by Internet pioneer Henry Spencer, it mentioned:

"3. Thou shalt cast all function arguments to the expected type if they are not of that type already, even when thou art convinced that this is unnecessary, lest they take cruel vengeance upon thee when thou least expect it." https://www.lysator.liu.se/c/ten-commandments.html

This was 36 years ago. Unfortunately this kind of bugs is still lurking inside many projects.

I'll send a patch to fix the problem, it's simply a matter of hunting for all usages of variadic functions and replacing 0 to NULL.

biergaizi commented 1 year ago

Just a single line of change is needed to fix the problem.

biergaizi commented 1 year ago

I've opened Pull Request #214 to fix this problem.

I'm not sure which branch to merge, I selected 4.9.3 only as a placeholder. Please tell me if an adjustment is needed.

gracicot commented 1 year ago

Wow!! Thank you so much. It seems the error message was not so far off!

biergaizi commented 1 year ago

It seems the error message was not so far off!

lol

gracicot commented 1 year ago

lol

exactly

biergaizi commented 1 year ago

My fix has been merged into the upstream as branch version/4.9.4. It's expected to be released as the stable version "in the next couple of days." Meanwhile, please use git branch version/4.9.4.