tool chain update for 23.05 release

[cproc]

My issue4827 branch [1] contains a first version of the update with binutils 2.40 and gcc 12.2.0.

TODO:

• RISC-V support
• ADA runtime update
• GDB update
• use C++20 by default

I'm going to create separate issues for the new errors which occurred so far when I tried to build Genode with the updated tools.

[1] https://github.com/cproc/genode/tree/issue4827

[nfeske]

That's just fantastic! @cproc, you made my day.

[chelmuth]

I just merged all compilation fixes to staging in preparation.

[chelmuth]

I tried to build the tool chain via ./tool/tool_chain x86 MAKE_JOBS=28 BUILD_LOCATION=/var/tmp/build-23.05 but got the following errors.

gpr-env.adb:1867:16: no candidate interpretations match the actuals:
gpr-env.adb:1867:16: missing argument for parameter "Separators" in call to "CREATE" declared at g-arrspl.ads:97, instance at g-strspl.ads:39
gpr-env.adb:1867:16: missing argument for parameter "Separators" in call to "CREATE" declared at g-arrspl.ads:83, instance at g-strspl.ads:39
gpr-env.adb:1867:16: context requires function call, found procedure name
gpr-env.adb:1867:53: unmatched actual "Mode" in call
gpr-env.adb:1868:16: ambiguous call to "Append"
gpr-env.adb:1868:16: possible interpretation at a-coinve.ads:230, instance at gpr-util.ads:39
gpr-env.adb:1868:16: possible interpretation at a-coinve.ads:234, instance at gpr-util.ads:39
gpr-env.adb:1868:25: "P" is undefined
gpr-env.adb:1884:19: no selector "Prepend_Vector" for private type "Ada.Containers.Indefinite_Vectors.Vector" from instance at gpr-util.ads:39
gpr-env.adb:1886:19: no selector "Append_Vector" for private type "Ada.Containers.Indefinite_Vectors.Vector" from instance at gpr-util.ads:39
gprbuild-link.adb:762:20: no selector "Append_Vector" for private type "Ada.Containers.Indefinite_Vectors.Vector" from instance at gpr-util.ads:39
gprbuild-link.adb:793:22: no selector "Append_Vector" for private type "Ada.Containers.Indefinite_Vectors.Vector" from instance at gprbuild.ads:165
gprbuild-link.adb:796:22: no selector "Append_Vector" for private type "Ada.Containers.Indefinite_Vectors.Vector" from instance at gprbuild.ads:165
gprbuild-link.adb:879:19: no selector "Append_Vector" for private type "Ada.Containers.Indefinite_Vectors.Vector" from instance at gprbuild.ads:165
gprbuild-link.adb:3364:37: no selector "Append_Vector" for private type "Ada.Containers.Indefinite_Vectors.Vector" from instance at gprbuild.ads:165
gprbuild-link.adb:3383:19: no selector "Append_Vector" for private type "Ada.Containers.Indefinite_Vectors.Vector" from instance at gprbuild.ads:165
gpr-env.adb:1867:16: no candidate interpretations match the actuals:
gpr-env.adb:1867:16: missing argument for parameter "Separators" in call to "CREATE" declared at g-arrspl.ads:97, instance at g-strspl.ads:39
gpr-env.adb:1867:16: missing argument for parameter "Separators" in call to "CREATE" declared at g-arrspl.ads:83, instance at g-strspl.ads:39
gpr-env.adb:1867:16: context requires function call, found procedure name
gpr-env.adb:1867:53: unmatched actual "Mode" in call
gpr-env.adb:1868:16: ambiguous call to "Append"
gpr-env.adb:1868:16: possible interpretation at a-coinve.ads:230, instance at gpr-util.ads:39
gpr-env.adb:1868:16: possible interpretation at a-coinve.ads:234, instance at gpr-util.ads:39
gpr-env.adb:1868:25: "P" is undefined
gpr-env.adb:1884:19: no selector "Prepend_Vector" for private type "Ada.Containers.Indefinite_Vectors.Vector" from instance at gpr-util.ads:39
gpr-env.adb:1886:19: no selector "Append_Vector" for private type "Ada.Containers.Indefinite_Vectors.Vector" from instance at gpr-util.ads:39
gnatmake: "/var/tmp/build-23.05/bootstrap/gprbuild/gpr/src/gpr-env.adb" compilation error
gnatmake: "/var/tmp/build-23.05/bootstrap/gprbuild/src/gprbuild-link.adb" compilation error
gnatmake: "/var/tmp/build-23.05/bootstrap/gprbuild/gpr/src/gpr-util.adb" compilation error
make: *** [tool/tool_chain:472: /var/tmp/build-23.05/bootstrap/install/bin/gprbuild] Error 4

Info about the Linux installation

• Ubuntu 20.04.5 LTS
  • gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
• GNATMAKE 9.4.0

[cproc]

@chelmuth: can you execute binaries from /var/tmp? I wonder if maybe the host tools are used to build gprbuild instead of the bootstrap tools.

[chelmuth]

The following succeeds, or did I get you wrong?

> /var/tmp/build-23.05/bootstrap/gcc/gcc/nm --version
GNU nm (GNU Binutils) 2.40
Copyright (C) 2023 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) any later version.
This program has absolutely no warranty.

[chelmuth]

Is the following correct or does it confirm your suspicion? Do you need more info?

> head /var/tmp/build-23.05/bootstrap/gprbuild/gprbuild.ali 
V "GNAT Lib v9"
P ZX

[cproc]

Is the following correct or does it confirm your suspicion? Do you need more info?

> head /var/tmp/build-23.05/bootstrap/gprbuild/gprbuild.ali 
V "GNAT Lib v9"
P ZX

In my Xubuntu 20.04 test VM it says

V "GNAT Lib v12"
P SS ZX

The tool_chain script prepends the bootstrap install location to the PATH environment variable, so normally the bootstrap tools should be used to build gprbuild. Since the execution permission of the bootstrap binaries is not the problem, maybe a bootstrap tool is missing for some reason? This is the output I get from ls -1 /var/tmp/build-23.05/bootstrap/install/bin:

addr2line
ar
as
c++
c++filt
cpp
elfedit
g++
gcc
gcc-ar
gccgo
gcc-nm
gcc-ranlib
gcov
gcov-dump
gcov-tool
gnat
gnatbind
gnatchop
gnatclean
gnatkr
gnatlink
gnatls
gnatmake
gnatname
gnatprep
gprbuild
gprclean
gprconfig
gprinstall
gprls
gprname
ld
ld.bfd
nm
objcopy
objdump
ranlib
readelf
size
strings
strip
x86_64-pc-linux-gnu-c++
x86_64-pc-linux-gnu-g++
x86_64-pc-linux-gnu-gcc
x86_64-pc-linux-gnu-gcc-12.2.0
x86_64-pc-linux-gnu-gcc-ar
x86_64-pc-linux-gnu-gccgo
x86_64-pc-linux-gnu-gcc-nm
x86_64-pc-linux-gnu-gcc-ranlib

[chelmuth]

> ls -1 /var/tmp/build-23.05/bootstrap/install/bin
addr2line*
ar*
as*
c++*
c++filt*
cpp*
elfedit*
g++*
gcc*
gcc-ar*
gcc-nm*
gcc-ranlib*
gcov*
gcov-dump*
gcov-tool*
ld*
ld.bfd*
nm*
objcopy*
objdump*
ranlib*
readelf*
size*
strings*
strip*
x86_64-pc-linux-gnu-c++*
x86_64-pc-linux-gnu-g++*
x86_64-pc-linux-gnu-gcc*
x86_64-pc-linux-gnu-gcc-12.2.0*
x86_64-pc-linux-gnu-gcc-ar*
x86_64-pc-linux-gnu-gcc-nm*
x86_64-pc-linux-gnu-gcc-ranlib*

[chelmuth]

Could it be I have an incomplete/inconsistent state as I started with ENABLE_FEATURES="c c++" and later extended to ENABLE_FEATURES="c c++ ada"?

[cproc]

Could it be I have an incomplete/inconsistent state as I started with ENABLE_FEATURES="c c++" and later extended to ENABLE_FEATURES="c c++ ada"?

That could be the reason. There is currently only a make dependency on the existence of g++ in the bootstrap install location.

[chelmuth]

I deleted /var/tmp/build-23.05/bootstrap and built from scratch without errors. Hooray :partying_face:

[cproc]

GDB is now updated to version 13.1 and the RISC-V tool chain is built with --with-arch=rv64imac_zicsr_zifencei.

[chelmuth]

Building gdb requires libgmp-dev on Ubuntu 20.04. mpfr and others seem optional.

[chelmuth]

Running make -C build/x86_64 run/log KERNEL=linux BOARD=linux yields some deterring warnings.

/usr/local/genode/tool/23.05/bin/../lib/gcc/x86_64-pc-elf/12.2.0/../../../../x86_64-pc-elf/bin/ld: warning: init_main_thread.o: missing .note.GNU-stack section implies executable stack
/usr/local/genode/tool/23.05/bin/../lib/gcc/x86_64-pc-elf/12.2.0/../../../../x86_64-pc-elf/bin/ld: NOTE: This behaviour is deprecated and will be removed in a future version of the linker
/usr/local/genode/tool/23.05/bin/genode-x86-ld: warning: test.o: missing .note.GNU-stack section implies executable stack
/usr/local/genode/tool/23.05/bin/genode-x86-ld: NOTE: This behaviour is deprecated and will be removed in a future version of the linker

Could we prevent this?

[chelmuth]

I'd also suggest the following to prevent warnings about hypervisor has a LOAD segment with RWX permissions.

+++ b/repos/base-nova/src/kernel/nova/target.mk
@@ -45,7 +45,8 @@ endif
 CC_CXX_WARN_STRICT = -Wextra -Weffc++ -Werror

 git_version      = $(shell cd $(NOVA_SRC_DIR) && (git rev-parse HEAD 2>/dev/null || echo 0) | cut -c1-7)
-CXX_LINK_OPT     = -Wl,--gc-sections -Wl,--warn-common -Wl,-static -Wl,-n -Wl,--defsym=GIT_VER=0x$(call git_version)
+CXX_LINK_OPT     = -Wl,--gc-sections -Wl,--warn-common -Wl,-static -Wl,-n -Wl,--defsym=GIT_VER=0x$(call git_version) \
+                   -Wl,--no-warn-rwx-segments
 LD_TEXT_ADDR     = # 0xc000000000 - when setting this 64bit compile fails because of relocation issues!! 
 LD_SCRIPT_STATIC = hypervisor.o

[cproc]

Building gdb requires libgmp-dev on Ubuntu 20.04. mpfr and others seem optional.

We could check with pkg-config on Ubuntu 20.04, but Ubuntu 18.04 does not have pkg-config support for libgmp. Is Ubuntu 18.04 support required?

[chelmuth]

We could check with pkg-config on Ubuntu 20.04, but Ubuntu 18.04 does not have pkg-config support for libgmp. Is Ubuntu 18.04 support required?

In my opinion, we can drop 18.04 support with the release.

[nfeske]

GCC 12.3 was released yesterday, which, according to the announcement, fixes 127 bugs.

https://gcc.gnu.org/pipermail/gcc/2023-May/241261.html

[cproc]

Fixup commit 3331dd4 adds the libgmp check to the tool_chain script.

[chelmuth]

Fixup commit 3331dd4 adds the libgmp check to the tool_chain script.

Nice. For the record, on Ubuntu 18.04 I can just run ./tool/tool_chain help GMP_OK= to skip the test.

[cproc]

I updated GCC to version 12.3.0 on my issue4827 branch.

[chelmuth]

I updated GCC to version 12.3.0 on my issue4827 branch.

Great, the server is already busy to build the new tool chain version ;-)

[ssumpf]

1a2b5c2 fixes JDK on my gcc-12 branch

[ssumpf]

987c821 and 5164038 fix stdcxx and libc for RISC-V.

[chelmuth]

1a2b5c2 fixes JDK on my gcc-12 branch 987c821 and 5164038 fix stdcxx and libc for RISC-V.

Merged to staging (and updated ports manually) to let the builder chew on it.

[chelmuth]

@ssumpf I also merged 5b78d276e7b9b76f095c41f8ea4deffe814dab94 en-passant as this approach looks reasonable to me. Is this okay as the commit effectively overwrite ARCH_DMA_MINALIGN for ARM too (where it is 128 in most cases)?

[ssumpf]

@ssumpf I also merged 5b78d27 en-passant as this approach looks reasonable to me. Is this okay as the commit effectively overwrite ARCH_DMA_MINALIGN for ARM too (where it is 128 in most cases)?

@chelmuth: Not sure, that's why I wrote to staff yesterday. We may also disable SIMD/SSE altogether. As far a I can tell it's not enabled at least for Linux x86.

[chelmuth]

We could set ARCH_DMA_MINALIGN only on pc to be safe...

[cnuke]

Rather than tinkering with the *_ALIGN settings I think it would be best to check the used flags by Linux and add the various '-no-* and so on flags to the corresponding import-*lx_emul.mk files. That brings us closer to the original compilation.

[chelmuth]

I fixed phone_manager.prg with genodelabs/genode-allwinner@76570433c56e6b2899a0d9b4033e4a7fb58eaf67.

[chelmuth]

Rather than tinkering with the *_ALIGN settings I think it would be best to check the used flags by Linux and add the various '-no-* and so on flags to the corresponding import-*lx_emul.mk files. That brings us closer to the original compilation.

Do I understand correctly that you suggest to inherit compiler options from Linux kernel, e.g. arch/x86/Makefile which almost expresses -mgeneral-regs-only by the following?

KBUILD_CFLAGS += -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx

[chelmuth]

Fixed zybo_gpio_demo_rgb.prg with genodelabs/genode-zynq@95cc6114b894b111ce3cbad83ba3284e0448785e.

[cnuke]

Rather than tinkering with the *_ALIGN settings I think it would be best to check the used flags by Linux and add the various '-no-* and so on flags to the corresponding import-*lx_emul.mk files. That brings us closer to the original compilation.

Do I understand correctly that you suggest to inherit compiler options from Linux kernel, e.g. arch/x86/Makefile which almost expresses -mgeneral-regs-only by the following?

KBUILD_CFLAGS += -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx

yes

[ssumpf]

bb67186 fixes RISC-V's dynamic linking.

[chelmuth]

yes

Looking for to your commits.

[chelmuth]

Fixed lx_fs.prg with 505d70f8dcfa9fccbacb80f76ae65800cedc0dc1.

[skalk]

@chelmuth commit 8d2be08 enables -mstrict-align for ARMv8 and disables FPU for ARMv7 and x86 to prevent the runtime alignment issues seen for example on i.MX 8MQ QSB. Please, give me a sign if you decide to apply it. I've to apply corresponding commits for genode-imx, genode-allwinner, genode-rpi, and genode-zynq, which depend on this one.

[chelmuth]

Let's do it today.

[cproc]

Fixup commit 3db325f updates the generated files for gcov and fixes the version mismatch error.

[chelmuth]

genodelabs/genode-imx@63bd0bbf08ec988658ed5e9d49218acc501c3ae7 fixes all error: call to ‘__bad_copy_from/to’` errors.

[cnuke]

I noticed ARCH_DMA_MINALIGN redefinition warnings during PinePhone testing and moved them into the pc repo (c8a48bc). It did not appear to break anything (running Sculpt on the Fuji6) an the PC-sde of things.

[cnuke]

yes

Looking for to your commits.

I noticed that my previous comment was somewhat dismissive. I briefly tinkered with those flags in the past while debugging the pc_wifi_drv on x86_32 and the reasoning was to make sure we stay closer to upstream compilation to avoid any potential short-comings be being more compliant for the contrib-code. That being said, from perspective that we run the drivers in user-space and the base library as well as the linker depend on the FPU anyway (I am not sure how intertwined it is with the vector-stuff) , making sure the driver works in this conditions is quite reasonable. I'll keep the topic in the back of my head but do not plan tackling that soon.

[cnuke]

@chelmuth please consider af04d22 for staging (addresses wireguard problems).

[chelmuth]

@chelmuth please consider af04d22 for staging (addresses wireguard problems).

Thanks, merged.

[atopia]

@chelmuth please consider atopia/genode-world@cc08750 for staging

[chelmuth]

@chelmuth please consider atopia/genode-world@cc08750 for staging

Thanks, it's in.

[cproc]

Commit 6f3d200 adds the Genode library name patterns to ld so that the -l option can find Genode libraries.

This removes the need to create dummy libraries when building some GNU applications, but also has the side-effect that the load order of the essential posix libraries (libc.lib.so, libm.lib.so, posix.lib.so and vfs.lib.so) can differ between binaries, for example if the GNU build system adds -lm before the other libraries on the link command line (seen with binutils, gcc and findutils so far), which causes problems with execve and fork. This could in principle also have happened before if a GNU application depended on an additional library with a name alphabetically sorted before libc. This problem can be solved by preloading posix.lib.so in scenarios where those applications are used with execve and fork, which commit 6bc544f does for the noux_tool_chain_auto.run script.

Commit f36b98d fixes the use of the pcre library in grep, which has not been used at all since the last grep update because the configure detection of the pcre library failed.

Commit 4499c9c removes the dummy libraries from noux-pkgs, except for libdl which does not exist as a Genode library.

[cproc]

Thinking a bit more about the library name pattern support, .abi.so should actually not be included because if such a library was found, the .abi.so name would end up in the binary, right? So symlinks to '.lib.so' always need to be created or ld would need to automatically rename .abi.so to .lib.so.

[cproc]

The .abi.so files have SONAME set as .lib.so, though, so I guess it would actually work correctly.