search

iains / gcc-darwin-arm64

GCC master branch for Darwin with experimental support for Arm64. Currently GCC-15.0.0 [September 2024]
GNU General Public License v2.0
268 stars 33 forks source link

`error: attempt to use poisoned "calloc"` when building a cross-compiler for x86_64-linux-musl host #84

Closed giordano closed 2 years ago

giordano commented 2 years ago

I'm trying to build on https://github.com/iains/gcc-darwin-arm64/commit/af646bebaceed617775b5465cf06cb5d270a16f4 a cross-compiler for build == host == x86_64-linux-musl (Alpine Linux 3.15) and target == aarch64-apple-darwin. I get the following error:

[12:57:48] libtool: compile:  x86_64-linux-musl-g++ -DHAVE_CONFIG_H -I. -I/workspace/srcdir/gcc-darwin-arm64/libcc1 -I /workspace/srcdir/gcc-darwin-arm64/libcc1/../include -I /workspace/srcdir/gcc-darwin-arm64/libcc1/../libgcc -I ../gcc -I/workspace/srcdir/gcc-darwin-arm64/libcc1/../gcc -I /workspace/srcdir/gcc-darwin-arm64/libcc1/../gcc/c -I/workspace/srcdir/gcc_build/./gmp -I/workspace/srcdir/gcc-darwin-arm64/gmp -I/workspace/srcdir/gcc_build/./mpfr/src -I/workspace/srcdir/gcc-darwin-arm64/mpfr/src -I/workspace/srcdir/gcc-darwin-arm64/mpc/src -I /workspace/srcdir/gcc-darwin-arm64/libcc1/../gcc/c-family -I /workspace/srcdir/gcc-darwin-arm64/libcc1/../libcpp/include -W -Wall -fvisibility=hidden -fcf-protection -g -O2 -MT libcc1plugin.lo -MD -MP -MF .deps/libcc1plugin.Tpo -c /workspace/srcdir/gcc-darwin-arm64/libcc1/libcc1plugin.cc  -fPIC -DPIC -o .libs/libcc1plugin.o
[12:57:49] In file included from /usr/include/pthread.h:30,
[12:57:49]                  from /usr/include/c++/10.3.1/x86_64-alpine-linux-musl/bits/gthr-default.h:35,
[12:57:49]                  from /usr/include/c++/10.3.1/x86_64-alpine-linux-musl/bits/gthr.h:148,
[12:57:49]                  from /usr/include/c++/10.3.1/ext/atomicity.h:35,
[12:57:49]                  from /usr/include/c++/10.3.1/memory:75,
[12:57:49]                  from /workspace/srcdir/gcc-darwin-arm64/libcc1/deleter.hh:23,
[12:57:49]                  from /workspace/srcdir/gcc-darwin-arm64/libcc1/rpc.hh:25,
[12:57:49]                  from /workspace/srcdir/gcc-darwin-arm64/libcc1/libcc1plugin.cc:67:
[12:57:49] /usr/include/sched.h:84:7: error: attempt to use poisoned "calloc"
[12:57:49]    84 | void *calloc(size_t, size_t);
[12:57:49]       |       ^
[12:57:49] /usr/include/sched.h:124:36: error: attempt to use poisoned "calloc"
[12:57:49]   124 | #define CPU_ALLOC(n) ((cpu_set_t *)calloc(1,CPU_ALLOC_SIZE(n)))
[12:57:49]       |                                    ^

Any idea of what's the problem or hints for where I should look at?

iains commented 2 years ago

OK from the snippet above I am seeing you are using 10.3 to build a cross to master - this is not something I test (because it does not work in the general case).

What I do (every week) is to:

The usual reason for that kind of error message when building GCC sources is that the source is directly including some <c++header> instead of using the mechanisms to get it included via "system.h".

However, I cannot tell very much since you have not included the configuration line you used for the cross.

Having said that, libcc1 is a host-side facility so it should be being built for x86_64-linux-musl, not aarch64-darwin.

giordano commented 2 years ago

Thanks for the prompt reply! You can see the full build script at https://github.com/JuliaPackaging/Yggdrasil/blob/8803707d466a07605788ac1b441cec0252b2c998/0_RootFS/gcc_common.jl#L277-L784, this is part of an effort for building GCC cross-compilers for multiple targets, so there will be unrelated stuff. This was successfully used in the past to build https://github.com/iains/gcc-darwin-arm64/commit/ccc57f4ed3feed697f17d3230786389b1b410af9, but @fxcoudert pointed out that was extremely old.

giordano commented 2 years ago

The usual reason for that kind of error message when building GCC sources is that the source is directly including some <c++header> instead of using the mechanisms to get it included via "system.h".

Like https://github.com/iains/gcc-darwin-arm64/blob/af646bebaceed617775b5465cf06cb5d270a16f4/libcc1/deleter.hh#L23 ?

iains commented 2 years ago

Hm. There's too much stuff there for me to be able to tell what the final command would look like.

As an aside - there's a bunch of "GCC doesn't do this right" comments in the configuration but (a) some of them seem wrong - at least GCC automatically adds LTO for me!! (b) you really should file bugs on the GCC bugzilla to get stuff fixed, that will reduce your maintenance burden....

Any chance you could find the actual configure line in the build transcript?

iains commented 2 years ago

The usual reason for that kind of error message when building GCC sources is that the source is directly including some <c++header> instead of using the mechanisms to get it included via "system.h".

Like

https://github.com/iains/gcc-darwin-arm64/blob/af646bebaceed617775b5465cf06cb5d270a16f4/libcc1/deleter.hh#L23

? Quite possibly although, of course, the version of master on which this is based has been bootstrapped on lots of systems - and the aarch64 port does not really make much change to libcc1 .

I would suggest you try to do a native bootstrap of latest branch - to make sure that works on its own. libcc1 is a host tool - so it would be being built with the gcc-10.3 compiler maybe that is the issue.

edit : when you do a bootstrap (which is what most devs do) - libcc1 is built with the stage 2 compiler - when you make a cross like the one you are doing - it is being built with the bootstrap compiler (so quite difference cases).

giordano commented 2 years ago

As an aside - there's a bunch of "GCC doesn't do this right" comments in the configuration but (a) some of them seem wrong - at least GCC automatically adds LTO for me!! (b) you really should file bugs on the GCC bugzilla to get stuff fixed, that will reduce your maintenance burden....

This single script is used to build GCC from v4.8.5 to 11.1, some stuff may have been fixed upstream but we'll still need to do manual tweaks to rebuild older versions anyway.

Any chance you could find the actual configure line in the build transcript?

https://github.com/JuliaPackaging/Yggdrasil/blob/8803707d466a07605788ac1b441cec0252b2c998/0_RootFS/gcc_common.jl#L757-L773

iains commented 2 years ago

Well .. I guess you know how all those variables substitute..

interesting that you specify the CC=/usr/bin/gcc CC_FOR_BUILD=/usr/bin/gcc CC_FOR_TARGET=${prefix}/bin/${COMPILER_TARGET}-gcc

but not the CXX= etc. [we bootstrap with C++ for some time now.]

(not sure if that could be an issue, but I specify both - although normally it is only necessary to specify CC/CXX for a bootstrap or a cross - and CC/CXX_FOR_BUILD when doing canadian or native crosses)

also you are building CC_FOR_TARGET, right?

(so I am not exactly sure what that will do when it comes to building the target libraries - which should be built using the "just built" compiler).

giordano commented 2 years ago

Well .. I guess you know how all those variables substitute..

sandbox:${WORKSPACE}/srcdir/gcc_build # env
SHELL=/bin/bash
host_includedir=/workspace/x86_64-linux-musl-cxx11/destdir/include
MACHTYPE=x86_64-linux-musl
CHARSET=UTF-8
prefix=/workspace/destdir
AS_FOR_TARGET=/workspace/destdir/bin/llvm-as
LD_FOR_TARGET=/workspace/destdir/bin/aarch64-apple-darwin20-ld
ac_cv_func_clock_gettime=no
proc_family=arm
PWD=/workspace/srcdir/gcc_build
bb_target=aarch64-apple-darwin20
exeext=
rust_host=x86_64-unknown-linux-musl
RANLIB_FOR_TARGET=/workspace/destdir/bin/llvm-ranlib
rust_target=aarch64-apple-darwin
V=true
includedir=/workspace/destdir/include
HISTFILE=/meta/.bash_history
ac_cv_func_mkostemp=no
libdir=/workspace/destdir/lib
bb_full_target=aarch64-apple-darwin20
AR_FOR_TARGET=/workspace/destdir/bin/llvm-ar
WORKSPACE=/workspace
TAPIDIR=/workspace/srcdir/apple-libtapi
host_prefix=/workspace/x86_64-linux-musl-cxx11/destdir
TERM=screen
USER=mose
SHLVL=1
HIDDEN_PS1=\[\]sandbox\[\]:\[\]${PWD//$WORKSPACE/$\{WORKSPACE\}}\[\] \$ 
dlext=dylib
NM_FOR_TARGET=/workspace/destdir/bin/llvm-nm
PS1=\[\]sandbox\[\]:\[\]${PWD//$WORKSPACE/$\{WORKSPACE\}}\[\] \$ 
FORCE_SANDBOX_MODE=unprivileged
SRC_NAME=GCCBootstrap
enable_libcilkrts=no
nbits=64
PATH=/workspace/destdir/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin
host_bindir=/workspace/x86_64-linux-musl-cxx11/destdir/bin
bindir=/workspace/destdir/bin
target=aarch64-apple-darwin20
nproc=24
host_libdir=/workspace/x86_64-linux-musl-cxx11/destdir/lib
OLDPWD=/workspace/srcdir/gcc_stage1
VERBOSE=true
_=/usr/bin/env

interesting that you specify the CC=/usr/bin/gcc CC_FOR_BUILD=/usr/bin/gcc CC_FOR_TARGET=${prefix}/bin/${COMPILER_TARGET}-gcc

but not the CXX= etc. [we bootstrap with C++ for some time now.]

Good point, but the C++ compiler being used and referenced in the compilation log in the first message above is the host compiler x86_64-linux-musl-g++

sandbox:${WORKSPACE}/srcdir/gcc_build # which x86_64-linux-musl-g++ 
/usr/bin/x86_64-linux-musl-g++
sandbox:${WORKSPACE}/srcdir/gcc_build # file $(which x86_64-linux-musl-g++)
/usr/bin/x86_64-linux-musl-g++: symbolic link to /usr/bin/g++
sandbox:${WORKSPACE}/srcdir/gcc_build # x86_64-linux-musl-g++ --version
x86_64-linux-musl-g++ (Alpine 10.3.1_git20211027) 10.3.1 20211027
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

also you are building CC_FOR_TARGET, right?

Yes, it's there:

sandbox:${WORKSPACE}/srcdir/gcc_build # ${prefix}/bin/${COMPILER_TARGET}-gcc --version
aarch64-apple-darwin20-gcc (GCC) 12.0.1 20220226 (experimental)
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

I would suggest you try to do a native bootstrap of latest branch - to make sure that works on its own. libcc1 is a host tool - so it would be being built with the gcc-10.3 compiler maybe that is the issue.

I tried https://github.com/iains/gcc-darwin-arm64/commit/1628ce542eb0697594f490fc24b9adfd3ebc3519 but I'm still running into the same error.

iains commented 2 years ago

also you are building CC_FOR_TARGET, right?

Yes, it's there:

sandbox:${WORKSPACE}/srcdir/gcc_build # ${prefix}/bin/${COMPILER_TARGET}-gcc --version
aarch64-apple-darwin20-gcc (GCC) 12.0.1 20220226 (experimental)
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

What I mean is that the build you are doing is intended to produce the target compiler and that compiler should be used to build the target libraries.

You are telling it to use a different (presumably older?) compiler to build the target libraries - which seems possibly problematical.

(nothing to do with the current problem, however)

I would suggest you try to do a native bootstrap of latest branch - to make sure that works on its own. libcc1 is a host tool - so it would be being built with the gcc-10.3 compiler maybe that is the issue.

I tried 1628ce5 but I'm still running into the same error.

So you are unable to do a native bootstrap of that branch on x86_64-linux-musl?

I do not have access to x86_64-linux-musl but will try on x86_64-linux-gnu.

iains commented 2 years ago

OK .. so I did this on x86_64-pc-linux-gnu

edit: using gcc-10 as the bootstrap compiler.

/src-patched/configure --prefix=/home/iains/gcc-master/gcc-12-0-1p --target=aarch64-apple-darwin20 --build=x86_64-pc-linux-gnu --enable-languages=c,c++,objc,obj-c++,fortran --with-pic --enable-host-shared  >conf.txt

make -j16 all-host

(because I do not have any binutils or sysroot on the Linux box for arm64 Darwin, the target stuff is not built - but libcc1 builds OK)

$ ls libcc1/.libs/*.so
libcc1/.libs/libcc1plugin.so  libcc1/.libs/libcc1.so  libcc1/.libs/libcp1plugin.so

gcc version 12.0.1 20220305 master-wip-apple-si revision r12-7593-gbc1fa4e2f04b

I built the branch that will be pushed later today - but AFAIK there's no significant difference in libcc1 from last week ...

... so I am not sure what issue you are facing ...

my recommendation is to make things less complicated:

  1. bootstrap the branch on x86_64-linux-muscl
  2. then use that to build the cross.
fxcoudert commented 2 years ago

It seems that the issue is specific to musl (won't occur with glibc), and occurs with upstream GCC (not specific to Iain's branch): there is a bug report at Gentoo that I believe to be the same issue https://bugs.gentoo.org/828580

iains commented 2 years ago

@fxcoudert thanks! one fewer thing to worry about ;)

fxcoudert commented 2 years ago

What is annoying is to see the Gentoo and Alpine are aware of an issue, but I can't find it reported at GCC 😢 … meaning it doesn't get fixed.

I've opened a GCC issue: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104799 It would probably be good to have a reproducer with exact configure and make command lines, but I don't have a musl box to test this.

giordano commented 2 years ago

Thanks for pointing me to the patch in Alpine Linux, that was very helpful! This patch

diff --git a/libcc1/libcc1plugin.cc b/libcc1/libcc1plugin.cc
index 12ab5a57c8d..6c9b70d4c70 100644
--- a/libcc1/libcc1plugin.cc
+++ b/libcc1/libcc1plugin.cc
@@ -17,6 +17,7 @@
    along with GCC; see the file COPYING3.  If not see
    <http://www.gnu.org/licenses/>.  */

+#include <pthread.h>
 #include <cc1plugin-config.h>

 #undef PACKAGE_NAME
diff --git a/libcc1/libcp1plugin.cc b/libcc1/libcp1plugin.cc
index 83dab7f58b1..4d0a6e0104e 100644
--- a/libcc1/libcp1plugin.cc
+++ b/libcc1/libcp1plugin.cc
@@ -18,6 +18,7 @@
    along with GCC; see the file COPYING3.  If not see
    <http://www.gnu.org/licenses/>.  */

+#include <pthread.h>
 #include <cc1plugin-config.h>

 #undef PACKAGE_NAME

which I also shared in the upstream bug report opened by @fxcoudert, does the trick for me.

However now I'm getting another error, see #85

giordano commented 2 years ago

BTW, I forgot to mention I also don't have a Musl machine, but I use BinaryBuilder.jl to build, which provides a build environment based on Alpine Linux. What I'm trying to do is to update the version of the GCC compiler for aarch64-apple-darwin for BinaryBuilder.

Outside of BinaryBuilder, you can easily get an Alpine Linux system by using Docker: docker run -ti --rm alpine:3.15

iains commented 2 years ago

heh .. the thing in shortest supply is time :)

thesamesam commented 2 years ago

What is annoying is to see the Gentoo and Alpine are aware of an issue, but I can't find it reported at GCC 😢 … meaning it doesn't get fixed.

I've opened a GCC issue: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104799 It would probably be good to have a reproducer with exact configure and make command lines, but I don't have a musl box to test this.

FWIW, the reason the patch hasn't been applied in Gentoo is the same reason it hasn't yet been reported upstream by us - we wanted to poke at it more and hadn't a chance. So it was planned. It currently blocks us moving forward with newer GCC.

Thanks for filing the bug!