Build musl toolchains

[DubbleClick]

Steps on how to build them on Alpine x86/aarch64:

git clone https://git.zv.io/toolchains/musl-cross-make.git
cd musl-cross-make
vi config,mak

config,.mak contents:

STAT = -static --static
FLAG = -g0 -Os -Wno-error

ifneq ($(NATIVE),)
COMMON_CONFIG += CC="$(HOST)-gcc ${STAT}" CXX="$(HOST)-g++ ${STAT}"
else
COMMON_CONFIG += CC="gcc ${STAT}" CXX="g++ ${STAT}"
endif

COMMON_CONFIG += CFLAGS="${FLAG}" CXXFLAGS="${FLAG}" LDFLAGS="${STAT}"
BINUTILS_CONFIG += --enable-gold=yes
GCC_CONFIG += --enable-static-pie --disable-cet
#  --enable-default-pie

CONFIG_SUB_REV = 888c8e3d5f7b
GCC_VER = 11.4.0
BINUTILS_VER = 2.40
MUSL_VER = 1.2.4
GMP_VER = 6.2.1
MPC_VER = 1.2.1
MPFR_VER = 4.2.0
LINUX_VER = 6.1.36

make TARGET=x86_64-linux-musl -j
(wait until it fails)
sed -i 's/poison calloc/poison/g' ./gcc-11.4.0/gcc/system.h
make TARGET=x86_64-linux-musl -j
make TARGET=x86_64-linux-musl install -j
tar -cxzf x86_64-musl-toolchain.tgz output/*

Might have to run make TARGET=x86_64-linux-musl -j a few times after replacing poison calloc with poison. It worked on the first try for me on x86, but took like 4-5 attempts on aarch64.

[crazywhalecc]

I tried on Docker Alpine with necessary build tools installed. But I got different error every time....

One example:

I have never compiled musl-cross-make, and this doesn't look like a stable build process. And it would be better if this process could be written as a Dockerfile.

[DubbleClick]

I'm not sure about automating it. It seems to throw errors randomly and the only way to help it is to use make again and again.

The author of musl.cc seems to have it working, though: https://musl.cc/#refs

make TARGET=x86_64-linux-musl -j
(wait until it fails)
sed -i 's/poison calloc/poison/g' ./gcc-11.4.0/gcc/system.h
make TARGET=x86_64-linux-musl -j
make TARGET=x86_64-linux-musl install -j
tar -cvzf x86_64-musl-toolchain.tgz output/*

This worked for me again on a fresh alpine x86_64 install, though.

[DubbleClick]

This worked for me:

config.mak

STAT = -static --static
FLAG = -g0 -Os -Wno-error

ifneq ($(NATIVE),)
COMMON_CONFIG += CC="$(HOST)-gcc ${STAT}" CXX="$(HOST)-g++ ${STAT}"
else
COMMON_CONFIG += CC="gcc ${STAT}" CXX="g++ ${STAT}"
endif

COMMON_CONFIG += CFLAGS="${FLAG}" CXXFLAGS="${FLAG}" LDFLAGS="${STAT}"

BINUTILS_CONFIG += --enable-gold=yes
GCC_CONFIG += --enable-static-pie --disable-cet
#--enable-default-pie

CONFIG_SUB_REV = 888c8e3d5f7b
GCC_VER = 11.4.0
BINUTILS_VER = 2.40
MUSL_VER = 1.2.4
GMP_VER = 6.2.1
MPC_VER = 1.2.1
MPFR_VER = 4.2.0

LINUX_VER = 6.1.36

Dockerfile

FROM alpine:3.18.4

RUN apk add --no-cache \
gcc g++ git make curl perl \
rsync patch wget libtool \
texinfo autoconf automake \
bison tar xz bzip2 zlib \
file binutils flex \
linux-headers libintl \
gettext gettext-dev icu-libs pkgconf \
pkgconfig icu-dev bash \
ccache libarchive-tools zip

WORKDIR /opt

RUN git clone https://git.zv.io/toolchains/musl-cross-make.git
WORKDIR /opt/musl-cross-make
COPY config.mak /opt/musl-cross-make

RUN make TARGET=x86_64-linux-musl -j; exit 0
RUN sed -i 's/poison calloc/poison/g' ./gcc-11.4.0/gcc/system.h
RUN make TARGET=x86_64-linux-musl -j
RUN make TARGET=x86_64-linux-musl install -j
RUN tar cvzf x86_64-musl-toolchain.tgz output/*

[crazywhalecc]

I don't know why, but using the Dockerfile above causes my OS to run out of memory and then kill all processes (including SSH, etc.). 😢

[crazywhalecc]

I debugged for a whole day and still failed to compile the result. This was a disaster. I fixed the out of memory issue, but one different random compilation error after another beats me.

[DubbleClick]

That's very strange, I tried it on both a rhel VM and the Alpine VM and it (the docker image) builds right away o.O

I can try it on my work vm but I can't imagine why it wouldn't work - docker is supposed to yield identical environments, after all.

[crazywhalecc]

To be honest, I kind of want to give up on this plan. After all, compiling in Alpine is perfect. But I will also try a new virtual machine environment tomorrow.

I also found a script that can build the alpine chroot environment with one click. It can be used, but the disadvantage is that you must copy the current spc source code or spc binary to the alpine rootfs, which Docker does not require.

[DubbleClick]

I think giving up on it is fair - but debian/rhel support should be dropped entirely then. There's no value in a spc that cannot build intl, it's maybe the most needed extension of all, I can't think of a single of my projects that doesn't require it.

Anyways, I will boot up my work vm in a bit.

Man, I started it almost 2 hours ago. It's still running. My work vm is REALLY slow. Oh, I figured out why. it's at 20gb ram usage for the docker container.

I think doing it like this with docker loads the entire source + build into ram rather than filesystem.

[DubbleClick]

500s on my home pc with 64gb ram.

Welcome to Alpine!

The Alpine Wiki contains a large amount of how-to guides and general
information about administrating Alpine systems.
See <https://wiki.alpinelinux.org/>.
alpine:~# cd /opt
alpine:/opt# ls
frankenphp       musl-cross-make  static-php-cli
alpine:/opt# mkdir musl-docker
alpine:/opt# cd musl-docker/
alpine:/opt/musl-docker# nano config.mak
alpine:/opt/musl-docker# echo "FROM alpine:3.18.4
>
> RUN apk add --no-cache \
> gcc g++ git make curl perl \
> rsync patch wget libtool \
> texinfo autoconf automake \
> bison tar xz bzip2 zlib \
> file binutils flex \
> linux-headers libintl \
> gettext gettext-dev icu-libs pkgconf \
> pkgconfig icu-dev bash \
> ccache libarchive-tools zip
>
> WORKDIR /opt
>
> RUN git clone https://git.zv.io/toolchains/musl-cross-make.git
> WORKDIR /opt/musl-cross-make
> COPY config.mak /opt/musl-cross-make
>
> RUN make TARGET=x86_64-linux-musl -j; exit 0
> RUN sed -i 's/poison calloc/poison/g' ./gcc-11.4.0/gcc/system.h
> RUN make TARGET=x86_64-linux-musl -j
> RUN make TARGET=x86_64-linux-musl install -j
> RUN tar cvzf x86_64-musl-toolchain.tgz output/*" > Dockerfile
alpine:/opt/musl-docker# apk add docker
fetch http://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.18/community/x86_64/APKINDEX.tar.gz
(1/16) Installing libseccomp (2.5.4-r2)
(2/16) Installing runc (1.1.7-r4)
(3/16) Installing containerd (1.7.2-r3)
(4/16) Installing containerd-openrc (1.7.2-r3)
(5/16) Installing libmnl (1.0.5-r1)
(6/16) Installing libnftnl (1.2.5-r1)
(7/16) Installing iptables (1.8.9-r2)
(8/16) Installing iptables-openrc (1.8.9-r2)
(9/16) Installing ip6tables (1.8.9-r2)
(10/16) Installing ip6tables-openrc (1.8.9-r2)
(11/16) Installing tini-static (0.19.0-r1)
(12/16) Installing docker-engine (23.0.6-r6)
(13/16) Installing docker-openrc (23.0.6-r6)
(14/16) Installing docker-cli (23.0.6-r6)
(15/16) Installing docker-cli-buildx (0.10.4-r9)
(16/16) Installing docker (23.0.6-r6)
Executing docker-23.0.6-r6.pre-install
Executing busybox-1.36.1-r4.trigger
OK: 999 MiB in 252 packages
alpine:/opt/musl-docker# docker build -t musl .
ERROR: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
alpine:/opt/musl-docker# service docker start
 * Caching service dependencies ...                                                                               [ ok ]
 * Mounting cgroup filesystem ...                                                                                 [ ok ]
 * /var/log/docker.log: creating file
 * /var/log/docker.log: correcting owner
 * Starting Docker Daemon ...                                                                                     [ ok ]
alpine:/opt/musl-docker# docker build -t musl .
[+] Building 497.3s (16/16) FINISHED
 => [internal] load .dockerignore                                                                                  0.0s
 => => transferring context: 2B                                                                                    0.0s
 => [internal] load build definition from Dockerfile                                                               0.0s
 => => transferring dockerfile: 697B                                                                               0.0s
 => [internal] load metadata for docker.io/library/alpine:3.18.4                                                   1.6s
 => [internal] load build context                                                                                  0.0s
 => => transferring context: 573B                                                                                  0.0s
 => [ 1/11] FROM docker.io/library/alpine:3.18.4@sha256:eece025e432126ce23f223450a0326fbebde39cdf496a85d8c016293f  0.4s
 => => resolve docker.io/library/alpine:3.18.4@sha256:eece025e432126ce23f223450a0326fbebde39cdf496a85d8c016293fc8  0.0s
 => => sha256:eece025e432126ce23f223450a0326fbebde39cdf496a85d8c016293fc851978 1.64kB / 1.64kB                     0.0s
 => => sha256:48d9183eb12a05c99bcc0bf44a003607b8e941e1d4f41f9ad12bdcc4b5672f86 528B / 528B                         0.0s
 => => sha256:8ca4688f4f356596b5ae539337c9941abc78eda10021d35cbc52659c74d9b443 1.47kB / 1.47kB                     0.0s
 => => sha256:96526aa774ef0126ad0fe9e9a95764c5fc37f409ab9e97021e7b4775d82bf6fa 3.40MB / 3.40MB                     0.3s
 => => extracting sha256:96526aa774ef0126ad0fe9e9a95764c5fc37f409ab9e97021e7b4775d82bf6fa                          0.1s
 => [ 2/11] RUN apk add --no-cache gcc g++ git make curl perl rsync patch wget libtool texinfo autoconf automake   6.9s
 => [ 3/11] WORKDIR /opt                                                                                           0.0s
 => [ 4/11] RUN git clone https://git.zv.io/toolchains/musl-cross-make.git                                         2.0s
 => [ 5/11] WORKDIR /opt/musl-cross-make                                                                           0.0s
 => [ 6/11] COPY config.mak /opt/musl-cross-make                                                                   0.0s
 => [ 7/11] RUN make TARGET=x86_64-linux-musl -j; exit 0                                                         218.9s
 => [ 8/11] RUN sed -i 's/poison calloc/poison/g' ./gcc-11.4.0/gcc/system.h                                        0.5s
 => [ 9/11] RUN make TARGET=x86_64-linux-musl -j                                                                 174.7s
 => [10/11] RUN make TARGET=x86_64-linux-musl install -j                                                           2.0s
 => [11/11] RUN tar cvzf x86_64-musl-toolchain.tgz output/*                                                       17.9s
 => exporting to image                                                                                            72.3s
 => => exporting layers                                                                                           72.3s
 => => writing image sha256:268e542108ed7d122c17d4337153d7e2f4bcbb7a81054baee1631218e5f7cce7                       0.0s
 => => naming to docker.io/library/musl                                                                            0.0s
alpine:/opt/musl-docker#

Ram usage up to 46gb (16-18gb for other programs/vms running on system). This may require a different approach to not require as much ram - it works fine without docker in my 8gb azure vm. So it has to be because of docker somehow.

[DubbleClick]

I think the best approach is to drop glibc based distro support entirely and instead offer a docker image that sets up alpine + spc, builds the extensions specified, zips up the buildroot directory and copies it to the host machine. What do you think?

Edit: Oh wow, my work VM finished and it only took 9571 seconds!

[crazywhalecc]

I think the best approach is to drop glibc based distro support entirely and instead offer a docker image that sets up alpine + spc, builds the extensions specified, zips up the buildroot directory and copies it to the host machine. What do you think?

I have no plans to deprecate glibc support for distributions such as Debian and RHEL. Because compiling with glibc has its special use: compiling non-purely static PHP to support dynamic loading of closed-source extensions. But now no one seems to need it in this, and I have not added this implementation to the main branch for the time being.

Perhaps if the non-alpine environment is detected and comes with C++ libraries or extensions, just prompt to use Docker or chroot to compile.

[crazywhalecc]

I set my VM to 32GB of RAM and it compiled successfully - peak usage was 23GB. Using less memory does not make the compilation successful, it just makes my virtual machine kill the process. 🤒

[crazywhalecc]

I noticed that you downloaded https://musl.cc/{$arch}-linux-musl-cross.tgz directly from musl.cc in the cplus branch. Is it the same as what we compiled?

[DubbleClick]

I noticed that you downloaded https://musl.cc/{$arch}-linux-musl-cross.tgz directly from musl.cc in the cplus branch. Is it the same as what we compiled?

Not exactly the same, the musl.cc one uses slightly older versions and compiled with --enable-default-pie. This is what I would do too, but there is currently a bug in php-intl that leads to a segmentation fault when it's compiled as position independent. All other extensions work fine as far as I could test.

https://github.com/php/php-src/issues/12442#issuecomment-1765420400

I have no plans to deprecate glibc support for distributions such as Debian and RHEL. Because compiling with glibc has its special use: compiling non-purely static PHP to support dynamic loading of closed-source extensions. But now no one seems to need it in this, and I have not added this implementation to the main branch for the time being.

But we couldn't do that when targeting musl or using musl-gcc like right now. I think what we should do then is use the musl toolchain produced with this¹ and get the C++ branch merged. Then at some point add a --dynamic-libc switch that uses the systems default gcc/g++. Or we do option ².

¹) I don't know docker well, but maybe someone who does would be able to minimise the ram usage and save to filesystem right away. Or we just need to provide 32gb of ram to compile this lol.

Perhaps if the non-alpine environment is detected and comes with C++ libraries or extensions, just prompt to use Docker or chroot to compile.

²) I believe either we use the musl toolchains built here or we only enable glibc's fake static compilation. C++ extensions are just essential and there's no point in using a musl-gcc wrapper if no C++ extensions are used. If --dynalic-libc is not used it should default to use an alpine/chroot environment to compile a static version, no matter if C++ extensions are used or not. Keep things as simple as possible.

So the options are:

1. Use musl-toolchain built with this. Build with $arch-linux-musl-gcc by default, unless --dynamic-libc is passed, then use system gcc/g++.
2. Use alpine docker/chroot environment to use spc on glibc. Use system-default gcc/g++. If --dynamic-libc is passed, use system-default gcc/g++ without changing chroot.
3. Keep musl-gcc. Build with musl-gcc if no c++ extensions are used. Build with docker-alpine if C++ extensions are used. Use system-default gcc/g++ if --dynamic-libc is passed.

Option 1 gives the best experience to users. Nothing but spc/php required. Option 2 is the easiest to maintain for us. Only alpine + glibc semi-dynamic compilation with system tools. But not as nice for users (need to install docker/set up chroot). Option 3 is the worst of all. Most to maintain and no benefit. What do you think? @crazywhalecc

[crazywhalecc]

After comprehensive consideration, I prefer option 1. 👋 But I think I need to add a few options to the download command first. For example --by-extensions=intl,redis,mongodb (only download libraries and extensions related to the specified extension), --without-cross-make (no need to download this under Alpine or compiling with glibc).

Option 1 will worsen the download speed in my network environment. musl-cross-toolchain.tgz is relatively large, and downloading large files from international servers in mainland China will be very slow unless a mirror or proxy is used. This is also the reason why I insisted on using package management to install tools before. Maybe in the future I can add a download address speed test function to select the alternative url or git repo to download large files, but this is not a main problem for this project and I can provide a local download config in the future.

[crazywhalecc]

I got the same error again and again when building in aarch64 alpine:

../../../src_binutils/gprofng/libcollector/unwind.c:4592:5: error: 'stack_base' undeclared (first use in this function); did you mean 'stack_t'?
 4592 |     stack_base = *sbase;
      |     ^~~~~~~~~~
      |     stack_t
../../../src_binutils/gprofng/libcollector/unwind.c:4604:16: error: 'pc' undeclared (first use in this function); did you mean 'putc'?
 4604 |   while (sp && pc)
      |                ^~
      |                putc
../../../src_binutils/gprofng/libcollector/unwind.c:4628:21: error: expected expression before ')' token
 4628 |       pc = ((__u64 *) sp)[1];
      |                     ^
make[6]: Leaving directory '/root/spc-musl/musl-cross-make/build/local/aarch64-linux-musl/obj_binutils/binutils'
make[5]: Leaving directory '/root/spc-musl/musl-cross-make/build/local/aarch64-linux-musl/obj_binutils/binutils'
make[4]: Leaving directory '/root/spc-musl/musl-cross-make/build/local/aarch64-linux-musl/obj_binutils/binutils'
../../../src_binutils/gprofng/libcollector/unwind.c:4629:12: error: expected ';' before 'old_sp'
 4629 |       __u64 old_sp = sp;
      |            ^~~~~~~
      |            ;
../../../src_binutils/gprofng/libcollector/unwind.c:4630:21: error: expected expression before ')' token
 4630 |       sp = ((__u64 *) sp)[0];
      |                     ^
../../../src_binutils/gprofng/libcollector/unwind.c:4631:16: error: 'old_sp' undeclared (first use in this function)
 4631 |       if (sp < old_sp)
      |                ^~~~~~
../../../src_binutils/gprofng/libcollector/unwind.c:4640:1: warning: control reaches end of non-void function [-Wreturn-type]
 4640 | }
      | ^

[crazywhalecc]

And I use a real arm64 VPS (32GB RAM), got this (sed -i 's/x86_64/aarch64/g' config.mak):

1.038 ../../../src_binutils/gprofng/libcollector/unwind.c:4592:5: error: 'stack_base' undeclared (first use in this function); did you mean 'stack_t'?
1.038  4592 |     stack_base = *sbase;
1.038       |     ^~~~~~~~~~
1.038       |     stack_t
1.040 ../../../src_binutils/gprofng/libcollector/unwind.c:4604:16: error: 'pc' undeclared (first use in this function); did you mean 'putc'?
1.040  4604 |   while (sp && pc)
1.040       |                ^~
1.040       |                putc
1.043 ../../../src_binutils/gprofng/libcollector/unwind.c:4628:21: error: expected expression before ')' token
1.043  4628 |       pc = ((__u64 *) sp)[1];
1.043       |                     ^
1.048 ../../../src_binutils/gprofng/libcollector/unwind.c:4629:12: error: expected ';' before 'old_sp'
1.048  4629 |       __u64 old_sp = sp;
1.048       |            ^~~~~~~
1.048       |            ;
1.050 ../../../src_binutils/gprofng/libcollector/unwind.c:4630:21: error: expected expression before ')' token
1.050  4630 |       sp = ((__u64 *) sp)[0];
1.050       |                     ^
1.054 make[6]: Leaving directory '/opt/musl-cross-make/build/local/aarch64-linux-musl/obj_binutils/binutils'
1.054 make[5]: Leaving directory '/opt/musl-cross-make/build/local/aarch64-linux-musl/obj_binutils/binutils'
1.054 ../../../src_binutils/gprofng/libcollector/unwind.c:4631:16: error: 'old_sp' undeclared (first use in this function)
1.054  4631 |       if (sp < old_sp)
1.054       |                ^~~~~~
1.054 make[4]: Leaving directory '/opt/musl-cross-make/build/local/aarch64-linux-musl/obj_binutils/binutils'
1.057 ../../../src_binutils/gprofng/libcollector/unwind.c:4640:1: warning: control reaches end of non-void function [-Wreturn-type]
1.057  4640 | }
1.057       | ^
1.058 make MULTILIB_OSDIRNAMES= INFO_DEPS= infodir= ac_cv_prog_lex_root=lex.yy MAKEINFO=false   all-am
1.062 make[7]: *** [Makefile:730: libgp_collector_la-unwind.lo] Error 1
1.062 make[7]: *** Waiting for unfinished jobs....

[DubbleClick]

You run make TARGET=aarch64-linux-musl -j right?

I didn't run into such a problem on neither alpine nor alma. But I can try resetting the vm.

What OS did you try on?

Option 1 will worsen the download speed in my network environment. musl-cross-toolchain.tgz is relatively large, and downloading large files from international servers in mainland China will be very slow unless a mirror or proxy is used. This is also the reason why I insisted on using package management to install tools before.

Hmm that makes sense, unfortunately it is not really feasible because of potential version mismatches. I think the download of it and musl-libc should stay in spc doctor, but an option --no-mcm can be added. When it stays in doctor it won't download or check it in alpine anyway.

[crazywhalecc]

You run make TARGET=aarch64-linux-musl -j right?

Yup.

What OS did you try on?

I've tried on:

• Debian 10 aarch64, installed official docker-ce, build with alpine:3.18.4 or alpine:edge (with 32GB physical RAM, 8 core).
• Debian 11 aarch64 virtual machine, installed official docker-ce, build the same alpine version (with 8GB physical RAM, 8 core, 32GB swapfile).
• Alpine edge aarch64 virtual machine, build with pure make command (with 8GB physical RAM, 8 core).

[DubbleClick]

Ooooh with docker. I don't have enough ram to try with docker 😅 Ok you just added alpine to the list. I will reinstall alpine and get it running again.

[crazywhalecc]

However, using non-docker to build will randomly report errors due to insufficient memory, and retrying more than a dozen times will have the same effect.

[crazywhalecc]

Latest build error:

``` ../../../src_binutils/gprofng/libcollector/unwind.c: In function '__collector_ext_return_address': ../../../src_binutils/gprofng/libcollector/unwind.c:236:34: error: '__u64' undeclared (first use in this function) 236 | context->uc_mcontext.sp = (__u64) __builtin_frame_address(0); \ | ^~~~~ ../../../src_binutils/gprofng/libcollector/unwind.c:490:3: note: in expansion of macro 'FILL_CONTEXT' 490 | FILL_CONTEXT ((&context)); | ^~~~~~~~~~~~ ../../../src_binutils/gprofng/libcollector/unwind.c:236:34: note: each undeclared identifier is reported only once for each function it appears in 236 | context->uc_mcontext.sp = (__u64) __builtin_frame_address(0); \ | ^~~~~ ../../../src_binutils/gprofng/libcollector/unwind.c:490:3: note: in expansion of macro 'FILL_CONTEXT' 490 | FILL_CONTEXT ((&context)); | ^~~~~~~~~~~~ ../../../src_binutils/gprofng/libcollector/unwind.c:236:41: error: expected ';' before '__builtin_frame_address' 236 | context->uc_mcontext.sp = (__u64) __builtin_frame_address(0); \ | ^~~~~~~~~~~~~~~~~~~~~~~ ../../../src_binutils/gprofng/libcollector/unwind.c:490:3: note: in expansion of macro 'FILL_CONTEXT' 490 | FILL_CONTEXT ((&context)); | ^~~~~~~~~~~~ ../../../src_binutils/gprofng/libcollector/unwind.c: In function '__collector_get_frame_info': ../../../src_binutils/gprofng/libcollector/unwind.c:236:34: error: '__u64' undeclared (first use in this function) 236 | context->uc_mcontext.sp = (__u64) __builtin_frame_address(0); \ | ^~~~~ ../../../src_binutils/gprofng/libcollector/unwind.c:528:7: note: in expansion of macro 'FILL_CONTEXT' 528 | FILL_CONTEXT (context); | ^~~~~~~~~~~~ ../../../src_binutils/gprofng/libcollector/unwind.c:236:41: error: expected ';' before '__builtin_frame_address' 236 | context->uc_mcontext.sp = (__u64) __builtin_frame_address(0); \ | ^~~~~~~~~~~~~~~~~~~~~~~ ../../../src_binutils/gprofng/libcollector/unwind.c:528:7: note: in expansion of macro 'FILL_CONTEXT' 528 | FILL_CONTEXT (context); | ^~~~~~~~~~~~ ../../../src_binutils/gprofng/libcollector/unwind.c: In function '__collector_getStackTrace': ../../../src_binutils/gprofng/libcollector/unwind.c:236:34: error: '__u64' undeclared (first use in this function) 236 | context->uc_mcontext.sp = (__u64) __builtin_frame_address(0); \ | ^~~~~ ../../../src_binutils/gprofng/libcollector/unwind.c:858:7: note: in expansion of macro 'FILL_CONTEXT' 858 | FILL_CONTEXT (context); | ^~~~~~~~~~~~ ../../../src_binutils/gprofng/libcollector/unwind.c:236:41: error: expected ';' before '__builtin_frame_address' 236 | context->uc_mcontext.sp = (__u64) __builtin_frame_address(0); \ | ^~~~~~~~~~~~~~~~~~~~~~~ ../../../src_binutils/gprofng/libcollector/unwind.c:858:7: note: in expansion of macro 'FILL_CONTEXT' 858 | FILL_CONTEXT (context); | ^~~~~~~~~~~~ ../../../src_binutils/gprofng/libcollector/unwind.c: In function 'stack_unwind': ../../../src_binutils/gprofng/libcollector/unwind.c:4582:3: error: unknown type name '__u64' 4582 | __u64 *lbuf = (void *) buf; | ^~~~~ ../../../src_binutils/gprofng/libcollector/unwind.c:4583:30: error: '__u64' undeclared (first use in this function) 4583 | int lsize = size / sizeof (__u64); | ^~~~~ ../../../src_binutils/gprofng/libcollector/unwind.c:4584:8: error: expected ';' before 'pc' 4584 | __u64 pc = context->uc_mcontext.pc; | ^~~ | ; ../../../src_binutils/gprofng/libcollector/unwind.c:4585:8: error: expected ';' before 'sp' 4585 | __u64 sp = context->uc_mcontext.sp; | ^~~ | ; ../../../src_binutils/gprofng/libcollector/unwind.c:4586:8: error: expected ';' before 'stack_base' 4586 | __u64 stack_base; | ^~~~~~~~~~~ | ; In file included from ../../../src_binutils/gprofng/libcollector/../common/hwcdrv.c:35, from ../../../src_binutils/gprofng/libcollector/libcol_hwcdrv.c:25: ../../../src_binutils/gprofng/libcollector/../common/cpuid.c:27:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '__get_cpuid' 27 | __get_cpuid (unsigned int op ATTRIBUTE_UNUSED, unsigned int *eax, | ^~~~~~~~~~~ ../../../src_binutils/gprofng/libcollector/unwind.c:4591:25: error: 'sp' undeclared (first use in this function) 4591 | if (sbase && *sbase > sp) | ^~ ../../../src_binutils/gprofng/libcollector/unwind.c:4592:5: error: 'stack_base' undeclared (first use in this function); did you mean 'stack_t'? 4592 | stack_base = *sbase; | ^~~~~~~~~~ | stack_t ../../../src_binutils/gprofng/libcollector/unwind.c:4604:16: error: 'pc' undeclared (first use in this function); did you mean 'putc'? 4604 | while (sp && pc) | ^~ | putc ../../../src_binutils/gprofng/libcollector/unwind.c:4628:21: error: expected expression before ')' token 4628 | pc = ((__u64 *) sp)[1]; | ^ ../../../src_binutils/gprofng/libcollector/unwind.c:4629:12: error: expected ';' before 'old_sp' 4629 | __u64 old_sp = sp; | ^~~~~~~ | ; ../../../src_binutils/gprofng/libcollector/unwind.c:4630:21: error: expected expression before ')' token 4630 | sp = ((__u64 *) sp)[0]; | ^ make MULTILIB_OSDIRNAMES= INFO_DEPS= infodir= ac_cv_prog_lex_root=lex.yy MAKEINFO=false all-am ../../../src_binutils/gprofng/libcollector/unwind.c:4631:16: error: 'old_sp' undeclared (first use in this function) 4631 | if (sp < old_sp) | ^~~~~~ ../../../src_binutils/gprofng/libcollector/unwind.c:4640:1: warning: control reaches end of non-void function [-Wreturn-type] 4640 | } | ^ ```

[DubbleClick]

I reset my VM and it's currently building.

Error 1:

Success:

Install success:

config.mak (I had to get rid of -static for some reason, but that's ok since we need to install musl-libc anyway):

#STAT = -static --static
FLAG = -g0 -Os -Wno-error

ifneq ($(NATIVE),)
COMMON_CONFIG += CC="$(HOST)-gcc ${STAT}" CXX="$(HOST)-g++ ${STAT}"
else
COMMON_CONFIG += CC="gcc ${STAT}" CXX="g++ ${STAT}"
endif

COMMON_CONFIG += CFLAGS="${FLAG}" CXXFLAGS="${FLAG}" LDFLAGS="${STAT}"

BINUTILS_CONFIG += --enable-gold=yes
GCC_CONFIG += --enable-static-pie --disable-cet
#--enable-default-pie

CONFIG_SUB_REV = 888c8e3d5f7b
GCC_VER = 11.4.0
BINUTILS_VER = 2.40
MUSL_VER = 1.2.4
GMP_VER = 6.2.1
MPC_VER = 1.2.1
MPFR_VER = 4.2.0
LINUX_VER = 6.1.36

[crazywhalecc]

After looking at it, it seems to be a compilation error of gprofng. I used BINUTILS_CONFIG += --enable-gold=yes --enable-gprofng=no directly and it seemed that the compilation was successful. Is it possible to solve the problem by removing gprofng?

BTW, my ARM server with 32GB cannot compile with Docker, it seems to require around 35GB memory under ARM architecture.

[DubbleClick]

I have no idea what gprofng is. I wish I had a arm machine to do better testing, but sadly only macs have adequate performance atm :(

Waiting on snapdragon 8cx4, maybe it will finally match the m3.

[crazywhalecc]

GPT answer 😯

[DubbleClick]

Oh profiler, we don't need that, but I wonder why it behaves differently on different systems. It would really be good if we could use docker to get an exact replica every time. But realistically, we don't have to build them that often, maybe once every few years.

[crazywhalecc]

You're right. Finally, I purchased a one-hour ARM VPS to compile it. This is not something we need to do often. We may need to write these things into documents later.