Recommended `--with-multilib-list=` list for GCC 13?

[ilg-ul]

I'm preparing for a new release of the xPack RISC-V Embedded GCC with GCC 13.2, and I'd appreciate any help in deciding on a reasonable list of multilibs.

The current list is:

• https://github.com/xpack-dev-tools/riscv-none-elf-gcc-xpack/blob/2fd6aead844120724290ef259fcbb1999ae4feed/scripts/versioning.sh#L61-L118

However, I saw discussions about the new extensions, like _zicsr (https://github.com/riscv-collab/riscv-gnu-toolchain/issues/1262), and about some changes in the multilib generator, thus I'm no longer sure that my list is still actual.

Any thoughts on this?

[TommyMurphyTM1234]

However, I saw discussions about the new extensions, like _zicsr (#1262)

I think that a key issue here is that previously the I (integer) base ISA subsumed Zicsr and Zifencei. But these were subsequently moved out of the base integer ISA to become independent/discrete extensions. (I'm not sure about E but I presume that the same applied there - i.e. E previously subsumed Zicsr and Zifencei but no longer does?).

Referring to the latest draft unprivileged specification:

• https://github.com/riscv/riscv-isa-manual/releases

to check when these changes happened:

• Zicsr: I can't see anything in the spec that clarifies
• Zifencei: Document Version 20190608-Base-Ratified

Whatever about the specification, as far as I know, the change was effected in GCC 12. This mailing list thread has some discussion about these changes:

• https://groups.google.com/a/groups.riscv.org/g/sw-dev/c/aE1ZeHHCYf4

So, prior to GCC 12 all architectures will have implicitly included the Zicsr and Zifencei extensions - but not after. This is why several issues arose here when people switched from GCC 11 to GCC 12 and started getting errors about the fact that their selected target arch/abi did not support the Zicsr and/or Zifencei extension.

So rvXXi prior to GCC 12 is equivalent to rvXXi_zicsr_zifencei from GCC 12 onwards.

It seems to me that in most cases in practice it's still necessary to have Zicsr enabled. I'm not so sure about Zifencei.

and about some changes in the multilib generator

I think that they involved some sort of generalisation of the mechanism that matches the selected arch/abi specified at compile time (using -march=... -mabi=...) to a suitable available multilib rather than relying solely (or at all?) on "reuse" patterns specified to multilib-generator. However I don't fully understand the changes and have not seen any documenation that clearly explains it. There is an Issue or PR here or GCC patch upstream that discusses this in some detail (which still didn't clarify things for me unfortunately) but at the moment I cannot locate it.

thus I'm no longer sure that my list is still actual. Any thoughts on this?

My opinion is that trying to have a single toolchain that supports all or most target arch/abi's via multilibs is foolhardy given the increasing number of RISC-V extensions and the number of possible combinations of same. I would be more inclined to try to assess what actual RISC-V targets are available and support those. However "malleable" targets such as simulators and soft IP/FPGA implementations, where arbitrary combinations of extensions can be enabled/disabled, may still represent a challenge.

[TommyMurphyTM1234]

BTW - if you look at issues such as this it seems to me that there are still outstanding anomalies with how canonicalised architecture strings are used and interpreted which can lead to problems when building or using the toolchain:

• https://github.com/riscv-collab/riscv-gnu-toolchain/issues/1126#issuecomment-1250679690

BTW - one example is that the default riscv-gnu-toolchain base arch/abi of rv64gc/lp64d gets expanded to rv64imafdc_zicsr_zifencei/lp64d in some places but IIRC not all. And strictly this is incorrect (nothing in gc implies Zicsr and Zifencei as of the aforementioned specification and GCC implementation changes) but maybe was intended as some sort of backwards compatibility band aid?

[TommyMurphyTM1234]

There is an Issue or PR here or GCC patch upstream that discusses this in some detail (which still didn't clarify things for me unfortunately) but at the moment I cannot locate it.

At the moment this is all that I can find in the GCC 13 release notes:

• https://gcc.gnu.org/gcc-13/changes.html#:~:text=Improves%20the%20multi%2Dlib%20selection%20mechanism%20for%20the%20bare%2Dmetal%20toolchain%20(riscv*%2Delf*).%20GCC%20will%20now%20automatically%20select%20the%20best%2Dfit%20multi%2Dlib%20candidate%20instead%20of%20requiring%20all%20possible%20reuse%20rules%20to%20be%20listed%20at%20build%20time.

Improves the multi-lib selection mechanism for the bare-metal toolchain (riscv-elf). GCC will now automatically select the best-fit multi-lib candidate instead of requiring all possible reuse rules to be listed at build time.

I think that this means that if the toolchain is configured as follows:

./configure --prefix=... --with-multilib-generator="rv32i-ilp32--"

then the resulting toolchain supports:

• rv64gc/lp64d by default
• rv32i/ilp32 via multilib

but RV32 architectures with "extension supersets" of rv32i will automatically also map to the rv32i/ilp32 multilib. For example:

• rv32im/ilp32
• rv32ima/ilp32
• etc.

should map to rv32i/ilp32 automatically. (I haven't actually tried exercising this mechanism myself yet).

Previously this would not happen unless the appropriate reuse mapping patterns were explicitly specified at toolchain configuration time and/or in the t-elf-multilib configuration file.

[ilg-ul]

Thank you, Tommy. Unfortunately I don't have the energy to fully understand these things... :-(

If there are no clear suggestions, I'll try to run a build with the same configuration and see if the result is still functional for my projects.

[TommyMurphyTM1234]

I forgot about this epic thread! :-D

• https://github.com/xpack-dev-tools/riscv-none-elf-gcc-xpack/issues/17

Apropos of that I suspect that you will need _zicsr variants of most or all of your existing multilibs - either instead of or in addition to the de facto list. However I'm not sure if that also extends to _zifencei and/or _zicsr_zifencei!

[TommyMurphyTM1234]

Apologies - inadvertently hit the close button so reopened it!

[TommyMurphyTM1234]

BTW - even apart from the whole (now) discrete Zicsr/Zifencei extension issue, users of the xPack toolchain might also expect to be able to use some or all of the various extensions ratified and supported upstream to date:

• GCC 12: https://gcc.gnu.org/gcc-12/changes.html

RISC-V

• Default ISA spec version was bump to 20191213, more detail see this announcement
• New ISA extension support for zba, zbb, zbc, zbs was added.
• New ISA extension support for vector and scalar crypto was added, only support architecture testing marco and -march= parsing.
• The option -mtune=thead-c906 is added to tune for T-HEAD c906 cores. libstdc++ no longer attempts to detect built-in atomics. Distributions that have out-of-tree workarounds for -latomic should check their ABIs again.

• GCC 13: https://gcc.gnu.org/gcc-13/changes.html

RISC-V

• Support for vector intrinsics as specified in version 0.11 of the RISC-V vector intrinsic specification, thanks Ju-Zhe Zhong from RiVAI for contributing most of implementation.
• Support for the following standard extensions has been added:
  • Zawrs
  • Zbkb
  • Zbkc
  • Zbkx
  • Zdinx
  • Zfinx
  • Zfh
  • Zfhmin
  • Zhinx
  • Zhinxmin
  • Zicbom
  • Zicbop
  • Zicboz
  • Zknd
  • Zkne
  • Zksed
  • Zksh
  • Zmmul
• Support for the following vendor extensions has been added:
  • XTheadBa
  • XTheadBb
  • XTheadBs
  • XTheadCmo
  • XTheadCondMov
  • XTheadFMemIdx
  • XTheadFmv
  • XTheadInt
  • XTheadMac
  • XTheadMemIdx
  • XTheadMemPair
  • XTheadSync
• The following new CPUs are supported through the -mcpu option (GCC identifiers in parentheses).
  • T-Head's XuanTie C906 (thead-c906).
• Improves the multi-lib selection mechanism for the bare-metal toolchain (riscv-elf). GCC will now automatically select the best-fit multi-lib candidate instead of requiring all possible reuse rules to be listed at build time.

And possibly more:

• https://wiki.riscv.org/display/HOME/Recently+Ratified+Extensions

and definitely more to come!

• https://wiki.riscv.org/display/HOME/Specification+Status

But - as I have said before - I think that trying to support most or all possible architectures/combinations of extensions in a single toolchain is foolhardy and maybe intractable. Maybe profiles and platforms will constrain this problem somewhat but I don't know.

• https://github.com/riscv/riscv-platform-specs

Going back to your original question I would consider extending this list:

        XBB_GCC_MULTILIB_LIST=${XBB_GCC_MULTILIB_LIST:-"\
          rv32e-ilp32e-- \
          rv32ea-ilp32e-- \
          rv32eac-ilp32e-- \
          rv32ec-ilp32e-- \
          rv32em-ilp32e-- \
          rv32ema-ilp32e-- \
          rv32emac-ilp32e-- \
          rv32emc-ilp32e-- \
          \
          rv32i-ilp32-- \
          rv32ia-ilp32-- \
          rv32iac-ilp32-- \
          rv32iaf-ilp32f-- \
          rv32iafc-ilp32f-- \
          rv32iafd-ilp32d-- \
          rv32iafdc-ilp32d-- \
          rv32ic-ilp32-- \
          rv32if-ilp32f-- \
          rv32ifc-ilp32f-- \
          rv32ifd-ilp32d-- \
          rv32ifdc-ilp32d-- \
          rv32im-ilp32-- \
          rv32ima-ilp32-- \
          rv32imaf-ilp32f-- \
          rv32imafc-ilp32f-- \
          rv32imafd-ilp32d-- \
          rv32imafdc-ilp32d-- \
          rv32imc-ilp32-- \
          rv32imf-ilp32f-- \
          rv32imfc-ilp32f-- \
          rv32imfd-ilp32d-- \
          rv32imfdc-ilp32d-- \
          \
          rv64i-lp64-- \
          rv64ia-lp64-- \
          rv64iac-lp64-- \
          rv64iaf-lp64f-- \
          rv64iafc-lp64f-- \
          rv64iafd-lp64d-- \
          rv64iafdc-lp64d-- \
          rv64ic-lp64-- \
          rv64if-lp64f-- \
          rv64ifc-lp64f-- \
          rv64ifd-lp64d-- \
          rv64ifdc-lp64d-- \
          rv64im-lp64-- \
          rv64ima-lp64-- \
          rv64imac-lp64-- \
          rv64imaf-lp64f-- \
          rv64imafc-lp64f-- \
          rv64imafd-lp64d-- \
          rv64imafdc-lp64d-- \
          rv64imc-lp64-- \
          rv64imf-lp64f-- \
          rv64imfc-lp64f-- \
          rv64imfd-lp64d-- \
          rv64imfdc-lp64d-- \
        "}

to this:

        XBB_GCC_MULTILIB_LIST=${XBB_GCC_MULTILIB_LIST:-"\
          rv32e-ilp32e-- \
          rv32ea-ilp32e-- \
          rv32eac-ilp32e-- \
          rv32ec-ilp32e-- \
          rv32em-ilp32e-- \
          rv32ema-ilp32e-- \
          rv32emac-ilp32e-- \
          rv32emc-ilp32e-- \
          \
          rv32e_zicsr_zifencei-ilp32e-- \
          rv32ea_zicsr_zifencei-ilp32e-- \
          rv32eac_zicsr_zifencei-ilp32e-- \
          rv32ec_zicsr_zifencei-ilp32e-- \
          rv32em_zicsr_zifencei-ilp32e-- \
          rv32ema_zicsr_zifencei-ilp32e-- \
          rv32emac_zicsr_zifencei-ilp32e-- \
          rv32emc_zicsr_zifencei-ilp32e-- \       
          \
          \
          rv32i-ilp32-- \
          rv32ia-ilp32-- \
          rv32iac-ilp32-- \
          rv32iaf-ilp32f-- \
          rv32iafc-ilp32f-- \
          rv32iafd-ilp32d-- \
          rv32iafdc-ilp32d-- \
          rv32ic-ilp32-- \
          rv32if-ilp32f-- \
          rv32ifc-ilp32f-- \
          rv32ifd-ilp32d-- \
          rv32ifdc-ilp32d-- \
          rv32im-ilp32-- \
          rv32ima-ilp32-- \
          rv32imaf-ilp32f-- \
          rv32imafc-ilp32f-- \
          rv32imafd-ilp32d-- \
          rv32imafdc-ilp32d-- \
          rv32imc-ilp32-- \
          rv32imf-ilp32f-- \
          rv32imfc-ilp32f-- \
          rv32imfd-ilp32d-- \
          rv32imfdc-ilp32d-- \
          \
          rv32i_zicsr_zifencei-ilp32-- \
          rv32ia_zicsr_zifencei-ilp32-- \
          rv32iac_zicsr_zifencei-ilp32-- \
          rv32iaf_zicsr_zifencei-ilp32f-- \
          rv32iafc_zicsr_zifencei-ilp32f-- \
          rv32iafd_zicsr_zifencei-ilp32d-- \
          rv32iafdc_zicsr_zifencei-ilp32d-- \
          rv32ic_zicsr_zifencei-ilp32-- \
          rv32if_zicsr_zifencei-ilp32f-- \
          rv32ifc_zicsr_zifencei-ilp32f-- \
          rv32ifd_zicsr_zifencei-ilp32d-- \
          rv32ifdc_zicsr_zifencei-ilp32d-- \
          rv32im_zicsr_zifencei-ilp32-- \
          rv32ima_zicsr_zifencei-ilp32-- \
          rv32imaf_zicsr_zifencei-ilp32f-- \
          rv32imafc_zicsr_zifencei-ilp32f-- \
          rv32imafd_zicsr_zifencei-ilp32d-- \
          rv32imafdc_zicsr_zifencei-ilp32d-- \
          rv32imc_zicsr_zifencei-ilp32-- \
          rv32imf_zicsr_zifencei-ilp32f-- \
          rv32imfc_zicsr_zifencei-ilp32f-- \
          rv32imfd_zicsr_zifencei-ilp32d-- \
          rv32imfdc_zicsr_zifencei-ilp32d-- \
          \
          \
          rv64i-lp64-- \
          rv64ia-lp64-- \
          rv64iac-lp64-- \
          rv64iaf-lp64f-- \
          rv64iafc-lp64f-- \
          rv64iafd-lp64d-- \
          rv64iafdc-lp64d-- \
          rv64ic-lp64-- \
          rv64if-lp64f-- \
          rv64ifc-lp64f-- \
          rv64ifd-lp64d-- \
          rv64ifdc-lp64d-- \
          rv64im-lp64-- \
          rv64ima-lp64-- \
          rv64imac-lp64-- \
          rv64imaf-lp64f-- \
          rv64imafc-lp64f-- \
          rv64imafd-lp64d-- \
          rv64imafdc-lp64d-- \
          rv64imc-lp64-- \
          rv64imf-lp64f-- \
          rv64imfc-lp64f-- \
          rv64imfd-lp64d-- \
          rv64imfdc-lp64d-- \
          \
          rv64i_zicsr_zifencei-lp64-- \
          rv64ia_zicsr_zifencei-lp64-- \
          rv64iac_zicsr_zifencei-lp64-- \
          rv64iaf_zicsr_zifencei-lp64f-- \
          rv64iafc_zicsr_zifencei-lp64f-- \
          rv64iafd_zicsr_zifencei-lp64d-- \
          rv64iafdc_zicsr_zifencei-lp64d-- \
          rv64ic_zicsr_zifencei-lp64-- \
          rv64if_zicsr_zifencei-lp64f-- \
          rv64ifc_zicsr_zifencei-lp64f-- \
          rv64ifd_zicsr_zifencei-lp64d-- \
          rv64ifdc_zicsr_zifencei-lp64d-- \
          rv64im_zicsr_zifencei-lp64-- \
          rv64ima_zicsr_zifencei-lp64-- \
          rv64imac_zicsr_zifencei-lp64-- \
          rv64imaf_zicsr_zifencei-lp64f-- \
          rv64imafc_zicsr_zifencei-lp64f-- \
          rv64imafd_zicsr_zifencei-lp64d-- \
          rv64imafdc_zicsr_zifencei-lp64d-- \
          rv64imc_zicsr_zifencei-lp64-- \
          rv64imf_zicsr_zifencei-lp64f-- \
          rv64imfc_zicsr_zifencei-lp64f-- \
          rv64imfd_zicsr_zifencei-lp64d-- \
          rv64imfdc_zicsr_zifencei-lp64d-- \
        "}

The rationale is that the rvXX*_zicsr_zifencei multilibs are the current equivalent of the older rvXX* ones that implicitly supported Zicsr and Zifencei. However, existing projects that select, say, rv32i/ilp32 will fail if they actually use Zicsr and/or Zifencei instructions - in which case they need to use rv32i_zicsr_zifencei/ilp32. (Note that the GCC 13 automatic multilib matching mechanism does not help here - I tried it).

FWIW I don't think that any hardware RISC-V implementation (whatever about simulators or soft IP FPGA implementations) does not implement Zicsr and Zifencei, many (if not most or all) will require the use of CSRs in startup/HAL code, and RISC-V OpenOCD currently assumes that the debug target supports the fencei instruction (see here).

[ilg-ul]

I need to study this more carefully, but my first question is why adding _zicsr_zifencei is not enough, and I have to keep the short definitions too?

[TommyMurphyTM1234]

I need to study this more carefully, but my first question is why adding _zicsr_zifencei is not enough, and I have to keep the short definitions too?

Because existing projects will be using, say, -march=rv32i -mabi=ilp32 and will fail to link if no rv32i/ilp32 multilib exists? Of course, if (as I think is a very strong possibility) these projects use CSR instructions (and/or fencei) then compilation will fail before link time anyway even if the rv32i/ilp32 multilib does exist. And such projects will not map to the rv32i_zicsr_zifencei/ilp32 multilib even with the new GCC 13 multilib matching.

It might make more sense to just have the _zicsr_zifencei versions of the multilibs but then existing projects and Makefiles will need to change (to ensure that Zicsr and Zifencei extensions are selected/specified) and you might also want to change the Eclipse Embedded CDT plugins to select these options by default for new projects?

[ilg-ul]

If I remember right, some previous builds based on the SiFive sources used a trick to define a full set of arch/abi pair, but redirect some to a limited set of libraries.

In other words, compile a single variant of the library with rv32i/ilp32, and also define rv32i_zicsr_zifencei/ilp32 to use the same library (assuming the libraries do not use the extra instructions).

I have to check if this is still possible with the GCC 13 multi-lib generator.

[TommyMurphyTM1234]

In other words, compile a single variant of the library with rv32i/ilp32, and also define rv32i_zicsr_zifencei/ilp32 to use the same library (assuming the libraries do not use the extra instructions).

I have to check if this is still possible with the GCC 13 multi-lib generator.

FWIW...

// ~/Downloads/hello.c
#include <stdio.h>

int main() 
{
    printf("Hello world\n");
    return 0;
}

git clone https://github.com/riscv-collab/riscv-gnu-toolchain
cd riscv-gnu-toolchain
git clone https://github.com/gcc-mirror/gcc -b releases/gcc-13 gcc-13

# Build a GCC 13 based RISC-V toolchain supporting rv64gc[_zicsr?]/lp64d by default and rv32i/ilp32 via multilib
./configure --prefix=`pwd`/installed-tools --with-multilib-generator="rv32i-ilp32--" --with-gcc-src=`pwd`/gcc-13 --disable-gdb
make

# Try to compile and link an rv32i_zicsr/ilp32 program
# Succeeds - links with the rv32i/ilp32 multilib
cd installed-tools/bin
./riscv64-unknown-elf-gcc -march=rv32i_zicsr -mabi=ilp32 ~/Downloads/hello.c -v
Using built-in specs.
COLLECT_GCC=./riscv64-unknown-elf-gcc
COLLECT_LTO_WRAPPER=/home/user/Downloads/riscv-gnu-toolchain/installed-tools/libexec/gcc/riscv64-unknown-elf/13.2.1/lto-wrapper
Target: riscv64-unknown-elf
Configured with: /home/user/Downloads/riscv-gnu-toolchain/gcc-13/configure --target=riscv64-unknown-elf --prefix=/home/user/Downloads/riscv-gnu-toolchain/installed-tools --disable-shared --disable-threads --enable-languages=c,c++ --with-pkgversion=g93c4226585c --with-system-zlib --enable-tls --with-newlib --with-sysroot=/home/user/Downloads/riscv-gnu-toolchain/installed-tools/riscv64-unknown-elf --with-native-system-header-dir=/include --disable-libmudflap --disable-libssp --disable-libquadmath --disable-libgomp --disable-nls --disable-tm-clone-registry --src=/home/user/Downloads/riscv-gnu-toolchain/gcc-13 --enable-multilib --with-multilib-generator=rv32i-ilp32-- --with-abi=lp64d --with-arch=rv64imafdc --with-tune=rocket --with-isa-spec=20191213 'CFLAGS_FOR_TARGET=-Os    -mcmodel=medlow' 'CXXFLAGS_FOR_TARGET=-Os    -mcmodel=medlow'
Thread model: single
Supported LTO compression algorithms: zlib
gcc version 13.2.1 20230822 (g93c4226585c)
COLLECT_GCC_OPTIONS='-march=rv32i_zicsr' '-mabi=ilp32' '-v' '-mtune=rocket' '-misa-spec=20191213' '-march=rv32i_zicsr' '-dumpdir' 'a-'
 /home/user/Downloads/riscv-gnu-toolchain/installed-tools/libexec/gcc/riscv64-unknown-elf/13.2.1/cc1 -quiet -v -imultilib rv32i/ilp32 hello.c -quiet -dumpdir a- -dumpbase hello.c -dumpbase-ext .c -march=rv32i_zicsr -mabi=ilp32 -mtune=rocket -misa-spec=20191213 -march=rv32i_zicsr -version -o /tmp/ccrYGFXr.s
GNU C17 (g93c4226585c) version 13.2.1 20230822 (riscv64-unknown-elf)
        compiled by GNU C version 11.4.0, GMP version 6.2.1, MPFR version 4.1.0, MPC version 1.2.1, isl version none
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring nonexistent directory "/home/user/Downloads/riscv-gnu-toolchain/installed-tools/riscv64-unknown-elf/usr/local/include"
ignoring duplicate directory "/home/user/Downloads/riscv-gnu-toolchain/installed-tools/riscv64-unknown-elf/include"
#include "..." search starts here:
#include <...> search starts here:
 /home/user/Downloads/riscv-gnu-toolchain/installed-tools/lib/gcc/riscv64-unknown-elf/13.2.1/include
 /home/user/Downloads/riscv-gnu-toolchain/installed-tools/lib/gcc/riscv64-unknown-elf/13.2.1/include-fixed
 /home/user/Downloads/riscv-gnu-toolchain/installed-tools/lib/gcc/riscv64-unknown-elf/13.2.1/../../../../riscv64-unknown-elf/include
End of search list.
Compiler executable checksum: d81a8fbeebe759d5f4a59ab9278ca799
COLLECT_GCC_OPTIONS='-march=rv32i_zicsr' '-mabi=ilp32' '-v' '-mtune=rocket' '-misa-spec=20191213' '-march=rv32i_zicsr' '-dumpdir' 'a-'
 /home/user/Downloads/riscv-gnu-toolchain/installed-tools/lib/gcc/riscv64-unknown-elf/13.2.1/../../../../riscv64-unknown-elf/bin/as -v --traditional-format -march=rv32i_zicsr -march=rv32i_zicsr -mabi=ilp32 -misa-spec=20191213 -o /tmp/ccjPVlKJ.o /tmp/ccrYGFXr.s
GNU assembler version 2.40.0 (riscv64-unknown-elf) using BFD version (GNU Binutils) 2.40.0.20230214
COMPILER_PATH=/home/user/Downloads/riscv-gnu-toolchain/installed-tools/libexec/gcc/riscv64-unknown-elf/13.2.1/:/home/user/Downloads/riscv-gnu-toolchain/installed-tools/libexec/gcc/riscv64-unknown-elf/13.2.1/:/home/user/Downloads/riscv-gnu-toolchain/installed-tools/libexec/gcc/riscv64-unknown-elf/:/home/user/Downloads/riscv-gnu-toolchain/installed-tools/lib/gcc/riscv64-unknown-elf/13.2.1/:/home/user/Downloads/riscv-gnu-toolchain/installed-tools/lib/gcc/riscv64-unknown-elf/:/home/user/Downloads/riscv-gnu-toolchain/installed-tools/lib/gcc/riscv64-unknown-elf/13.2.1/../../../../riscv64-unknown-elf/bin/
LIBRARY_PATH=/home/user/Downloads/riscv-gnu-toolchain/installed-tools/lib/gcc/riscv64-unknown-elf/13.2.1/rv32i/ilp32/:/home/user/Downloads/riscv-gnu-toolchain/installed-tools/lib/gcc/riscv64-unknown-elf/13.2.1/../../../../riscv64-unknown-elf/lib/rv32i/ilp32/:/home/user/Downloads/riscv-gnu-toolchain/installed-tools/riscv64-unknown-elf/lib/rv32i/ilp32/:/home/user/Downloads/riscv-gnu-toolchain/installed-tools/lib/gcc/riscv64-unknown-elf/13.2.1/:/home/user/Downloads/riscv-gnu-toolchain/installed-tools/lib/gcc/riscv64-unknown-elf/13.2.1/../../../../riscv64-unknown-elf/lib/:/home/user/Downloads/riscv-gnu-toolchain/installed-tools/riscv64-unknown-elf/lib/
COLLECT_GCC_OPTIONS='-march=rv32i_zicsr' '-mabi=ilp32' '-v' '-mtune=rocket' '-misa-spec=20191213' '-march=rv32i_zicsr' '-dumpdir' 'a.'
 /home/user/Downloads/riscv-gnu-toolchain/installed-tools/libexec/gcc/riscv64-unknown-elf/13.2.1/collect2 -plugin /home/user/Downloads/riscv-gnu-toolchain/installed-tools/libexec/gcc/riscv64-unknown-elf/13.2.1/liblto_plugin.so -plugin-opt=/home/user/Downloads/riscv-gnu-toolchain/installed-tools/libexec/gcc/riscv64-unknown-elf/13.2.1/lto-wrapper -plugin-opt=-fresolution=/tmp/cc6stDhb.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgloss -plugin-opt=-pass-through=-lgcc --sysroot=/home/user/Downloads/riscv-gnu-toolchain/installed-tools/riscv64-unknown-elf -melf32lriscv /home/user/Downloads/riscv-gnu-toolchain/installed-tools/lib/gcc/riscv64-unknown-elf/13.2.1/../../../../riscv64-unknown-elf/lib/rv32i/ilp32/crt0.o /home/user/Downloads/riscv-gnu-toolchain/installed-tools/lib/gcc/riscv64-unknown-elf/13.2.1/rv32i/ilp32/crtbegin.o -L/home/user/Downloads/riscv-gnu-toolchain/installed-tools/lib/gcc/riscv64-unknown-elf/13.2.1/rv32i/ilp32 -L/home/user/Downloads/riscv-gnu-toolchain/installed-tools/lib/gcc/riscv64-unknown-elf/13.2.1/../../../../riscv64-unknown-elf/lib/rv32i/ilp32 -L/home/user/Downloads/riscv-gnu-toolchain/installed-tools/riscv64-unknown-elf/lib/rv32i/ilp32 -L/home/user/Downloads/riscv-gnu-toolchain/installed-tools/lib/gcc/riscv64-unknown-elf/13.2.1 -L/home/user/Downloads/riscv-gnu-toolchain/installed-tools/lib/gcc/riscv64-unknown-elf/13.2.1/../../../../riscv64-unknown-elf/lib -L/home/user/Downloads/riscv-gnu-toolchain/installed-tools/riscv64-unknown-elf/lib /tmp/ccjPVlKJ.o -lgcc --start-group -lc -lgloss --end-group -lgcc /home/user/Downloads/riscv-gnu-toolchain/installed-tools/lib/gcc/riscv64-unknown-elf/13.2.1/rv32i/ilp32/crtend.o
COLLECT_GCC_OPTIONS='-march=rv32i_zicsr' '-mabi=ilp32' '-v' '-mtune=rocket' '-misa-spec=20191213' '-march=rv32i_zicsr' '-dumpdir' 'a.'

cd ../..
make distclean
rm -rf installed-tools

# Build a GCC 13 based RISC-V toolchain supporting rv64gc[_zicsr?]/lp64d by default and rv32i_zicsr_zifencei/ilp32 via multilib
./configure --prefix=`pwd`/installed-tools --with-multilib-generator="rv32i_zicsr_zifencei-ilp32--" --with-gcc-src=`pwd`/gcc-13  --disable-gdb
make

# Try to compile and link an rv32i/ilp32 program
# Fails
cd installed-tools/bin
./riscv64-unknown-elf-gcc -march=rv32i -mabi=ilp32 ~/Downloads/hello.c -v
riscv64-unknown-elf-gcc: fatal error: Cannot find suitable multilib set for '-march=rv32i'/'-mabi=ilp32'
compilation terminated.

# Try to compile and link an rv32i_zicsr/ilp32 program
# Fails
./riscv64-unknown-elf-gcc -march=rv32i_zicsr -mabi=ilp32 ~/Downloads/hello.c
riscv64-unknown-elf-gcc: fatal error: Cannot find suitable multilib set for '-march=rv32i_zicsr'/'-mabi=ilp32'
compilation terminated.

# Try to compile and link an rv32i_zifencei/ilp32 program
# Fails
./riscv64-unknown-elf-gcc -march=rv32i_zfencei -mabi=ilp32 ~/Downloads/hello.c
riscv64-unknown-elf-gcc: fatal error: Cannot find suitable multilib set for '-march=rv32i_zifencei'/'-mabi=ilp32'
compilation terminated.

# Try to compile and link an rv32i_zicsr_zifencei/ilp32 program
# Succeeds
./riscv64-unknown-elf-gcc -march=rv32i_zicsr_zfencei -mabi=ilp32 ~/Downloads/hello.c

# Try to compile and link an rv32i_zifencei_zicsr/ilp32 program
# Succeeds
./riscv64-unknown-elf-gcc -march=rv32i_zfencei_zicsr -mabi=ilp32 ~/Downloads/hello.c

[ilg-ul]

The multilib-generator reads:

# Each argument to this script is of the form
#  <primary arch>-<abi>-<additional arches>-<extensions>
# Example 1:
#  rv32imafd-ilp32d-rv32g-c,v
# means that, in addition to rv32imafd, these configurations can also use the
# rv32imafd-ilp32d libraries: rv32imafdc, rv32imafdv, rv32g, rv32gc, rv32gv
#
# Example 2:
#  rv32imafd-ilp32d--c*b
# means that, in addition to rv32imafd, these configurations can also use the
# rv32imafd-ilp32d libraries: rv32imafdc-ilp32d, rv32imafdb-ilp32d,
#                             rv32imafdcb-ilp32d

So it should be possible for the compiler to accept more configurations with various combinations of extensions, and use a common library for all.

[TommyMurphyTM1234]

I'm not sure that those "reuse" patterns necessarily still work as documented in the aftermath of the changes to GCC 13's multilib handling and automatic reuse matching.

Note that they don't work as documented for the Linux toolchain:

• https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111065

@kito-cheng - maybe you can comment here perhaps?

So it should be possible for the compiler to accept more configurations with various combinations of extensions, and use a common library for all.

Even if they did still work as documented then such matching normally involves some loss of functionality. For example:

# means that, in addition to rv32imafd, these configurations can also use the
# rv32imafd-ilp32d libraries: rv32imafdc, rv32imafdv, rv32g, rv32gc, rv32gv

Mapping rv32gc/ilp32d onto rv32imafd/ilp32d means that the linked startup/utility code and libraries will not take advantage of the target's hardware support for the C (compressed) extension which is almost certainly suboptimal.

I've said it before but personally I don't like any sort of reuse mapping like this - or the GCC 13 built-in version where it's unclear what exactly is going on - and prefer that if one is targeting a specific arch/abi then one links startup/utility code and libraries compiled specifically for that arch/abi and not something less optimal. But this makes it difficult to create a one size fits all toolchain that tries to cater for most or all possible architectures and combinations of extensions.

[ilg-ul]

I did a limited build with the following

        XBB_GCC_MULTILIB_LIST=${XBB_GCC_MULTILIB_LIST:-"\
          rv32emac-ilp32e-- \
          rv32ima-ilp32--zicsr*zifencei \
          rv64imac-lp64-- \
        "}

Regardless if the multilibs include zicsr or zifencei, all compile commands pass:

/home/ilg/Work/xpack-dev-tools/riscv-none-elf-gcc-xpack.git/build/linux-x64/application/bin/riscv-none-elf-gcc -march=rv32ima -mabi=ilp32 hello.c
/home/ilg/Work/xpack-dev-tools/riscv-none-elf-gcc-xpack.git/build/linux-x64/application/bin/riscv-none-elf-gcc -march=rv32ima_zicsr -mabi=ilp32 hello.c 
/home/ilg/Work/xpack-dev-tools/riscv-none-elf-gcc-xpack.git/build/linux-x64/application/bin/riscv-none-elf-gcc -march=rv32ima_zifencei -mabi=ilp32 hello.c 
/home/ilg/Work/xpack-dev-tools/riscv-none-elf-gcc-xpack.git/build/linux-x64/application/bin/riscv-none-elf-gcc -march=rv32ima_zifencei_zicsr -mabi=ilp32 hello.c 
/home/ilg/Work/xpack-dev-tools/riscv-none-elf-gcc-xpack.git/build/linux-x64/application/bin/riscv-none-elf-gcc -march=rv64imac -mabi=lp64 hello.c 
/home/ilg/Work/xpack-dev-tools/riscv-none-elf-gcc-xpack.git/build/linux-x64/application/bin/riscv-none-elf-gcc -march=rv64imac_zicsr -mabi=lp64 hello.c 
/home/ilg/Work/xpack-dev-tools/riscv-none-elf-gcc-xpack.git/build/linux-x64/application/bin/riscv-none-elf-gcc -march=rv64imac_zicsr_zifencei -mabi=lp64 hello.c 
/home/ilg/Work/xpack-dev-tools/riscv-none-elf-gcc-xpack.git/build/linux-x64/application/bin/riscv-none-elf-gcc -march=rv32emac -mabi=ilp32e hello.c 
/home/ilg/Work/xpack-dev-tools/riscv-none-elf-gcc-xpack.git/build/linux-x64/application/bin/riscv-none-elf-gcc -march=rv32emac_zicsr -mabi=ilp32e hello.c 
/home/ilg/Work/xpack-dev-tools/riscv-none-elf-gcc-xpack.git/build/linux-x64/application/bin/riscv-none-elf-gcc -march=rv32imac -mabi=ilp32 hello.c 
/home/ilg/Work/xpack-dev-tools/riscv-none-elf-gcc-xpack.git/build/linux-x64/application/bin/riscv-none-elf-gcc -march=rv32imac_zicsr -mabi=ilp32 hello.c 
/home/ilg/Work/xpack-dev-tools/riscv-none-elf-gcc-xpack.git/build/linux-x64/application/bin/riscv-none-elf-gcc -march=rv32imac_zicsr_zifencei -mabi=ilp32 hello.c 
/home/ilg/Work/xpack-dev-tools/riscv-none-elf-gcc-xpack.git/build/linux-x64/application/bin/riscv-none-elf-gcc -march=rv32imac_zifencei -mabi=ilp32 hello.c 

There is only a single folder with each library:

$ tree -L 1 /home/ilg/Work/xpack-dev-tools/riscv-none-elf-gcc-xpack.git/build/linux-x64/application/riscv-none-elf/lib/rv*
/home/ilg/Work/xpack-dev-tools/riscv-none-elf-gcc-xpack.git/build/linux-x64/application/riscv-none-elf/lib/rv32emac
└── ilp32e
/home/ilg/Work/xpack-dev-tools/riscv-none-elf-gcc-xpack.git/build/linux-x64/application/riscv-none-elf/lib/rv32ima
└── ilp32
/home/ilg/Work/xpack-dev-tools/riscv-none-elf-gcc-xpack.git/build/linux-x64/application/riscv-none-elf/lib/rv64imac
└── lp64

I would say that the behaviour is as expected.

For readability I will probably update my long list and suffix all lines with --zicsr*zifencei, but this seems not mandatory.

@kito-cheng, can you confirm my findings?

@TommyMurphyTM1234, can you test the same configuration on your machine?

[TommyMurphyTM1234]

I did a limited build with the following

        XBB_GCC_MULTILIB_LIST=${XBB_GCC_MULTILIB_LIST:-"\
          rv32emac-ilp32e-- \
          rv32ima-ilp32--zicsr*zifencei \
          rv64imac-lp64-- \
        "}

Regardless if the multilibs include zicsr or zifencei, all compile commands pass:

Do your test programs contain any Zicsr and/or Zifencei instructions?

I would say that the behaviour is as expected.

I don't really understand what you mean.

@TommyMurphyTM1234, can you test the same configuration on your machine?

I'm not clear on what exactly you want me to test?

[TommyMurphyTM1234]

rv32ima-ilp32--zicsr*zifencei

I guess that this is probably OK. When targeting, say, rv32ima_zicsr/ilp32 the startup/utility object files and libraries for rv32ima/ilp32 will be linked. These will not contain any Zicsr instructions but the user's code can contain such instructions.

This is different to, say, rv32ima-ilp32--c where the resulting linked program will be suboptimal because the startup/utility object files and libraries linked (rv32ima/ilp32) will not take advantage of compressed instructions.

[ilg-ul]

Do your test programs contain any Zicsr and/or Zifencei instructions?

No. But if the -march passed to the compiler, I expect it to accept these instructions.

I don't really understand what you mean.

I mean that regardless if the --zicsr*zifencei is present or not in the multilb configurations, the presence of any combination of _zicrs and _zifencei on the compiler line is accepted, and the correct library is linked.

I'm not clear on what exactly you want me to test?

Run a build with those multilibs, and invoke the compiler with those combinations of options.

When targeting, say, rv32ima_zicsr/ilp32 the startup/utility object files and libraries for rv32ima/ilp32 will be linked. These will not contain any Zicsr instructions but the user's code can contain such instructions.

Right. And this is the expected behaviour.

[TommyMurphyTM1234]

I mean that regardless if the --zicsr*zifencei is present or not in the multilb configurations, the presence of any combination of _zicrs and _zifencei on the compiler line is accepted, and the correct library is linked.

So in the absence of the "reuse" patterns for --zicsr*zifencei I guess that the changes in GCC 13 for multilib matching are kicking in and selecting the most suitable available multilib? And these changes should address the issue mentioned here?

• https://github.com/xpack-dev-tools/riscv-none-elf-gcc-xpack/issues/17

Run a build with those multilibs, and invoke the compiler with those combinations of options.

You mean a build of the xPack RISC-V GCC toolchain? Unfortunately I'm not set up to do that easily right now.

[ilg-ul]

You mean a build of the xPack RISC-V GCC toolchain?

No, the toolchain you already built with this repo. Change the multilb defs to that 3 line list and rebuild.

[TommyMurphyTM1234]

You mean a build of the xPack RISC-V GCC toolchain?

No, the toolchain you already built with this repo. Change the multilb defs to that 3 line list and rebuild.

Ah - OK - I can try that.

[TommyMurphyTM1234]

FWIW - based on these results it looks like the reuse pattern in rv32ima-ilp32--zicsr*zifencei is not actually needed after all?

git clone https://github.com/riscv-collab/riscv-gnu-toolchain
cd riscv-gnu-toolchain
git clone https://github.com/gcc-mirror/gcc -b releases/gcc-13 gcc-13
./configure --prefix=`pwd`/installed-tools --with-gcc-src=`pwd`/gcc-13 --with-multilib-generator="rv32emac-ilp32e--;rv32ima-ilp32--;rv64imac-lp64--"

cd installed-tools/bin
cp ~/Downloads/hello.c .

# All of the following compile and link OK
./riscv64-unknown-elf-gcc -march=rv32ima -mabi=ilp32 hello.c
./riscv64-unknown-elf-gcc -march=rv32ima_zicsr -mabi=ilp32 hello.c
./riscv64-unknown-elf-gcc -march=rv32ima_zifencei -mabi=ilp32 hello.c
./riscv64-unknown-elf-gcc -march=rv32ima_zicsr_zifencei -mabi=ilp32 hello.c
./riscv64-unknown-elf-gcc -march=rv32ima_zifencei_zicsr -mabi=ilp32 hello.c
./riscv64-unknown-elf-gcc -march=rv32imac -mabi=ilp32 hello.c
./riscv64-unknown-elf-gcc -march=rv32imaf -mabi=ilp32 hello.c
./riscv64-unknown-elf-gcc -march=rv32imad -mabi=ilp32 hello.c
./riscv64-unknown-elf-gcc -march=rv32imafdc -mabi=ilp32 hello.c
./riscv64-unknown-elf-gcc -march=rv32g -mabi=ilp32 hello.c
./riscv64-unknown-elf-gcc -march=rv32gc -mabi=ilp32 hello.c
./riscv64-unknown-elf-gcc -march=rv32gc_zicsr -mabi=ilp32 hello.c

# These fail
./riscv64-unknown-elf-gcc -march=rv32i -mabi=ilp32 hello.c
riscv64-unknown-elf-gcc: fatal error: Cannot find suitable multilib set for '-march=rv32i'/'-mabi=ilp32'
compilation terminated.
./riscv64-unknown-elf-gcc -march=rv32im -mabi=ilp32 hello.c
riscv64-unknown-elf-gcc: fatal error: Cannot find suitable multilib set for '-march=rv32im'/'-mabi=ilp32'
compilation terminated.
./riscv64-unknown-elf-gcc -march=rv32ia -mabi=ilp32 hello.c
riscv64-unknown-elf-gcc: fatal error: Cannot find suitable multilib set for '-march=rv32ia'/'-mabi=ilp32'
compilation terminated.
./riscv64-unknown-elf-gcc -march=rv32imaf -mabi=ilp32f hello.c
riscv64-unknown-elf-gcc: fatal error: Cannot find suitable multilib set for '-march=rv32imaf_zicsr'/'-mabi=ilp32f'
compilation terminated.

[ilg-ul]

They are not the same combinations, the idea was to try exactly the same definitions available in the multilib list, both with and without the _z* extensions, to ack that they are accepted regardless of the presence of the _z* extensions in the multilib list.

[TommyMurphyTM1234]

I'm doing that next - please give me some time!

[ilg-ul]

I did a test with 12.3, and it seems that for that version the presence of --zicsr*zifencei is significant, without it combinations like -march=rv64imac_zicsr_zifencei -mabi=lp64 fail.

So I'll keep the extended definitions.

[TommyMurphyTM1234]

Ok - same results as above using this configuration:

git clone https://github.com/riscv-collab/riscv-gnu-toolchain
cd riscv-gnu-toolchain
git clone https://github.com/gcc-mirror/gcc -b releases/gcc-13 gcc-13
./configure --prefix=`pwd`/installed-tools --with-gcc-src=`pwd`/gcc-13 --with-multilib-generator="rv32emac-ilp32e--;rv32ima-ilp32--zicsr*zifencei;rv64imac-lp64--"

Observations:

1. These results seem to imply that the ...zicsr*zifencei reuse pattern is not needed/is redundant.
2. It could be that this reuse pattern is totally ignored anyway because the GCC 13 built-in reuse mechanism always takes precendence - @kito-cheng should be able to clarify

[TommyMurphyTM1234]

I did a test with 12.3, and it seems that for that version the presence of --zicsr*zifencei is significant, without it combinations like -march=rv64imac_zicsr_zifencei -mabi=lp64 fail.

So I'll keep the extended definitions.

That makes sense if/when you are ever building GCC 12 or earlier (maybe for updates to the xPack releases of old GCC toolchains?). But it seems like redundant and potentially confusing verbiage for GCC 13 onwards given that the built-in multilib matching seems to take precedence and possibly renders the multilib-generator reuse patterns redundant?

[ilg-ul]

That makes sense if/when you are ever building GCC 12 or earlier (maybe for updates to the xPack releases of old GCC toolchains?)

Yes, GNU maintains old GCC toolchains for 3 more years (12.1 & 12.2 in 2022, 12.3 in 2023, 12.4 in 2024, 12.5 in 2025) and I plan to do the same. For example now, after 13.2, I'll release 12.3. Older versions had different issues requiring patches and are more difficult to maintain.

[TommyMurphyTM1234]

For readability I will probably update my long list and suffix all lines with --zicsr*zifencei, but this seems not mandatory.

Is this what you have decided in the end?

What about other extensions not already covered by your multilib list?

[ilg-ul]

Is this what you have decided in the end?

This is the current configuration used to build 13.2 (expected to be released tomorrow):

• https://github.com/xpack-dev-tools/riscv-none-elf-gcc-xpack/blob/8f9583ac566a65ddae5635bd4bbd6a94c362cc7e/scripts/versioning.sh#L64-L122

What about other extensions not already covered by your multilib list?

What about them? Did you find any problems while using them?

[TommyMurphyTM1234]

What about them? Did you find any problems while using them?

Not so far as I have tried them (working from the lists here) but only by virtue of the fact that the rv32ima/ilp32 objects/libs get linked and they do not benefit from/take advantage of the additional extensions specified. So, so far, it's functional but suboptimal.

[ilg-ul]

so far, it's functional but suboptimal

I'm not familiar with those new extensions, but I guess they are of little value for newlib or the system libraries, so the toolchain can be built without them and remain pretty useful for the majority of use cases.

For special cases, those who need them can always build their own toolchains.

[TommyMurphyTM1234]

I'm not familiar with those new extensions, but I guess they are of little value for newlib or the system libraries, so the toolchain can be built without them and remain pretty useful for the majority of use cases.

OK - that's probably reasonable alright.

[TommyMurphyTM1234]

Can this issue be closed now?