search

nodejs / build

Better build and test infra for Node.
507 stars 166 forks source link

move minimum supported gcc to gcc 12 for Node.js 23 #3806

Closed richardlau closed 3 weeks ago

richardlau commented 4 months ago

gcc no longer support gcc 10. I think it's time we plan to move up.

Due to compiler issues, we've already had to move Linux on s390x (LinuxONE) builds to use gcc 12: https://github.com/nodejs/build/issues/3630 (FWIW the Red Hat team maintaining the V8 ports to Power and z also use gcc 12.)

My proposal is to move to gcc 12 as not all platforms that we support that we use gcc to build on have gcc 13 yet (e.g. AIX, IBM i). It would be easier to have the same baseline across all supported platforms, although it is not strictly necessarily (in the past we have had the minimum vary by platform, but in practice that forces Node.js to be compatible with the lowest out of the set).

Compilation of main is currently broken on gcc 12.2 (although not on RHEL 8) but would be fixed by landing https://github.com/nodejs/node/pull/53728.

For releases on Linux we'd continue to use RHEL 8 (now in maintenance but good through to 2029) and use gcc-toolset-12 instead of gcc-toolset-10. This would mean we keep compatibility with glibc 2.28 and libstdc++ 6.0.25 (the base in RHEL 8) and Node.js 23 would continue to run in the same places that Node.js 22, 20 and 18 does. The exception to this is likely to be 32-bit arm (armv7l) which we use a custom cross-compiler for (as RHEL doesn't support armv7l). We would either need to generate a new gcc 12 cross-compiler (and check what that would mean in terms of run-time compatibility) or perhaps discuss whether armv7l is still a platform we continue to supply official binaries for.

targos commented 3 months ago

I opened https://github.com/nodejs/node/pull/54081 to change the warning threshold in Node.js.

It includes a temporary commit that makes the build exit with code 1 when the version is too low so we can easily assess the impact on CI:

https://ci.nodejs.org/job/node-test-pull-request/60690/

targos commented 2 months ago

New CI with error if gcc<12: https://ci.nodejs.org/job/node-test-commit/73486/

targos commented 2 months ago

Looks like we need to select gcc-12 on ubuntu2204:

targos commented 2 months ago

RHEL8:

targos commented 2 months ago

smartos: https://ci.nodejs.org/job/node-test-commit-smartos/56305/nodes=smartos20-64/console

More tricky as it doesn't seem to have gcc-12 installed

targos commented 2 months ago

RHEL9:

targos commented 2 months ago

arm cross-compiler: https://ci.nodejs.org/job/node-cross-compile/49474/

richardlau commented 2 months ago

arm cross-compiler: https://ci.nodejs.org/job/node-cross-compile/49474/

FWIW I'm trying (so far without success 😞) to update the cross compiler used.

I've generated a new gcc 12 based cross-compiler using crosstools-ng: https://github.com/richardlau/rpi-newer-crosstools/commits/gcc-12.3.0-glibc-2.28/

Unfortunately I'm getting undefined symbol errors when attempting to cross-compile for 32-bit arm in a RHEL 8 x64 container with gcc 12 host compiler (via gcc-toolset-12):

/opt/rh/gcc-toolset-12/root/usr/libexec/gcc/x86_64-redhat-linux/12/ld: /home/nodejs/node/out/Release/torque: hidden symbol `_ZNSt7__cxx1112basic_stringIwSt11char_traitsIwESaIwEE9_M_mutateEjjPKwj' isn't defined
/opt/rh/gcc-toolset-12/root/usr/libexec/gcc/x86_64-redhat-linux/12/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
make[1]: *** [tools/v8_gypfiles/torque.host.mk:216: /home/nodejs/node/out/Release/torque] Error 1
make[1]: *** Waiting for unfinished jobs....

My suspicion is that there's a mismatch between the version of libstdc++ in the cross-compiler vs in RHEL 8 (for the host compiler). I'm attempting (building at the moment) to see what happens in a RHEL 9 container (which should have a newer libstdc++ but may still be behind the one from the cross-compiler since RHEL 9 is based on gcc 11).

If that doesn't work (or even if it does), other options are:

richardlau commented 2 months ago

I'm attempting (building at the moment) to see what happens in a RHEL 9 container (which should have a newer libstdc++ but may still be behind the one from the cross-compiler since RHEL 9 is based on gcc 11).

Update: I got a binary using a RHEL 9 x64 container with gcc-toolset-12 as the host compiler and x64-gcc-12.3.0-glibc-2.28 from https://github.com/richardlau/rpi-newer-crosstools/commits/gcc-12.3.0-glibc-2.28/ which looks like a 32-bit arm binary:

bash-5.1$ /opt/rpi-newer-crosstools/x64-gcc-12.3.0-glibc-2.28/arm-rpi-linux-gnueabihf/bin/arm-rpi-linux-gnueabihf-objdump -f out/Release/node

out/Release/node:     file format elf32-littlearm
architecture: arm, flags 0x00000112:
EXEC_P, HAS_SYMS, D_PAGED
start address 0x009c4378

bash-5.1$

I'll do more checks next week (like verifying the binary runs on armv7🙂 and checking compatibility). 🤞

richardlau commented 2 months ago

I'll do more checks next week (like verifying the binary runs on armv7🙂 and checking compatibility). 🤞

The resultant binary works in

and fails (due to libstdc++) in

So it looks like we can retain support for 32-bit arm by using a RHEL 9 container with the generated gcc 12 cross-compiler . This looks like the quickest/least effort change:

Alternative, and more work, would be to move to compiling the 32-bit arm binary in a 32-bit arm container. This would mean:

richardlau commented 1 month ago

I've opened https://github.com/nodejs/build/pull/3889 to add Ansible changes to build 32-bit ARM binaries with the updated gcc 12 cross compiler.

richardlau commented 1 month ago

Looks like we need to select gcc-12 on ubuntu2204:

select-compiler PR for ubuntu2204: https://github.com/nodejs/build/pull/3890

targos commented 1 month ago

For SmartOS I saw that we have hosts with newer versions but they are not used by the Jenkins job

richardlau commented 1 month ago

For SmartOS I saw that we have hosts with newer versions but they are not used by the Jenkins job

Being tracked under https://github.com/nodejs/build/issues/3731 as part of the Equinix migration.

targos commented 1 month ago

New try: https://ci.nodejs.org/job/node-test-commit/74227/

targos commented 1 month ago

Only SmartOS remains.

richardlau commented 1 month ago

For the new MNX hosted SmartOS machines which are meant to replace the equinix_mnx ones, it looks like SmartOS 23 has gcc 13 by default:

$ ssh test-mnx-smartos23-x64-1 gcc --version
gcc (GCC) 13.2.0
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$

while SmartOS 21 is 10:

$ ssh test-mnx-smartos21-x64-1 gcc --version
gcc (GCC) 10.4.0
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$

So perhaps this will be taken care of when the build issues on the new machines are resolved in https://github.com/nodejs/build/issues/3731? cc @jperkin @ryanaslett

richardlau commented 3 weeks ago

Only SmartOS remains.

I'm going to close this issue -- we have https://github.com/nodejs/build/issues/3731 for tracking the migration of the SmartOS hosts and the gcc update should be taken care of by updating to SmartOS 23.