powerpc64: allow querying ELF ABI version

[q66]

There are two ABIs on ppc64, ELFv1 (legacy) and ELFv2 (introduced a couple years back, used by default for little endian). Rust currently has logic in it that for the musl libc, ELFv2 is used on big endian and ELFv1 is used otherwise. This is inadequate, as you can use ELFv2 as well with glibc. In my distribution, both little and big endian systems are supported, with glibc and musl, and they are all set up for ELFv2.

In order to achieve that, Rust in the distribution is patched so that it will always properly use ELFv2 ABI. However, this is a potential problem for some crates that for example need to deal with assembly code; they have no way to query the ABI version. For endianness, they can do like #[cfg(target_endian = "big")]. For ABI, there is no equivalent (in C, you can do #if _CALL_ELF == 2).

I can look into implementing this, but I need to know what would be the best syntax and way to implement this. As I see it, it would need a new field in the target options, but this would of course be specific to ppc64, and ABI on ppc64 in e.g. gcc is treated more like a list than a string (you can also have -mabi=altivec, -mabi=ibmlongdouble and so on, in addition to -mabi=elfv2), so I was wondering if something more generic/extensible would be better.

Of course, eventually it'd be best to also allow setting it when compiling, and make it properly pass everything down to llvm, but that's less of a concern, for now I just need crates to be able to read it.

[q66]

Actually, this could easily be implemented as a string in TargetOptions, matching llc -target-abi=<value>. So there is no need for it to be a list, considering LLVM already handles it like this. Then it could be exposed as target_abi cfg. I can try making a proof of concept implementation.

[cuviper]

Rust currently has logic in it that for the musl libc, ELFv2 is used on big endian and ELFv1 is used otherwise.

It's actually that all musl is ELFv2, as is other little-endian, while big-endian uses ELFv1.

https://github.com/rust-lang/rust/blob/cfdc84a009020c59e53e4039beae22eb59e41685/src/librustc_target/abi/call/powerpc64.rs#L127-L134

And here's what GCC has to say about those mabi=elfv1/2 settings: https://gcc.gnu.org/onlinedocs/gcc-9.1.0/gcc/RS_002f6000-and-PowerPC-Options.html#index-mabi_003delfv1

-mabi=elfv1

Change the current ABI to use the ELFv1 ABI. This is the default ABI for big-endian PowerPC 64-bit Linux. Overriding the default ABI requires special system support and is likely to fail in spectacular ways.

-mabi=elfv2

Change the current ABI to use the ELFv2 ABI. This is the default ABI for little-endian PowerPC 64-bit Linux. Overriding the default ABI requires special system support and is likely to fail in spectacular ways.

LLVM has a -target-abi option, but AFAICS we're not using it, and it defaults the same way as GCC:

https://github.com/rust-lang/llvm-project/blob/84abffda0e03b03c62bdfc3cfeda1c2cf1f88c85/llvm/lib/Target/PowerPC/PPCTargetMachine.cpp#L199-L202

So I wonder if the little bit of ABI handling we have for musl is actually enough...

[q66]

It's not enough (in my distro I use external LLVM, which is patched to use ELFv2 correctly, and for bootstrap executables that use internal LLVM i have that patched as well). Yes, all musl uses ELFv2. My point was more about that glibc can also use ELFv2 on big endian (and in my distro it does). That means Rust programs that rely on for example calling into external assembler cannot rely on the triple to tell which ABI version it will be, they need some other way of looking it up.

I'm working on a proof of concept implementation of this which will also include using -target-abi correctly.

[q66]

Anyway, my current design is something like the following:

1) TargetOptions gets a new field pub target_abi: Option<String>;matching llc -target-abi. This ABI would be passed when target_abi is not a None. 2) Most target specs will continue being as they are, the powerpc64(le) ones will set target_abi with a "sane" default, i.e. ELFv2 for LE, ELFv2 for musl, ELFv1 for BE+glibc. In the future, it could be possible to make this adjustable, so that rustc can be configured to use a different ABI, similarly to -mabi in GCC. For now, I would continue patching this in my distro, there are some other bits in the powerpc64-linux-gnu spec that I already need to patch anyway, like using full RELRO. 3) Remove those checks from powerpc64.rs, instead rely on the target_abi field from TargetOptions. 4) Provide a new #[cfg(target_abi = "elfv2")] or #[cfg(target_abi = "elfv1")] to be read by Rust programs, conditionally, depending on if the target_abi field of TargetOptions contains a string (if it contains a None, it will not be provided, which will be most platforms and will account for future expansion; for now ppc64 is enough).

[cuviper]

That design sounds good to me.

The only point that might be controversial is exposing this in cfg. For example, the float ABI of arm-unknown-linux-gnueabi vs arm-unknown-linux-gnueabihf is not exposed -- you can only tell by the extra target_feature. I think cfg(target_abi) does make sense though.

[q66]

That is true, but that's ARM, where you have the ABI defined as a part of the triple. Meanwhile, on POWER, you have a single triple environment, _CALL_ELF in C (this macro is required to be defined by the ELFv2 ABI spec itself) and programs are expected to check that.

So this would be pretty much _CALL_ELF's Rust counterpart. This also IMO makes sense, because both ELFv1 and ELFv2 binaries will run on the same kernel in the same environment, and they can technically interact with each other, as long as they're not being explicitly linked together. Additionally, for example FreeBSD only has a powerpc64-unknown-freebsd target triple, and they don't seem like they have any intentions of actually changing that, they are expecting software to account for _CALL_ELF correctly (FreeBSD is currently ELFv1, but it can be compiled as ELFv2 too, the only blocker is currently that they still use an ancient toolchain for POWER, so they need either Clang or modern GCC first).

[bdragon28]

Yeah, FreeBSD will be switching to Big Endian ELFv2 during the current (13-CURRENT) development cycle. Progress is being tracked at https://wiki.freebsd.org/powerpc/llvm-elfv2 and is moving forward at a rapid pace.

FreeBSD detection would generally done by caring about the version as well (the whole triple is powerpc64-unknown-freebsd13.0 and while it currently means "ELFv1" by default, it will mean "ELFv2" after flag day. powerpc64-unknown-freebsd12.0 and down can be considered to be ELFv1 and powerpc64-unknown-freebsd14.0 and up can be considered to be ELFv2. powerpc64-unknown-freebsd13.0 is the only ambiguous one.)

As you mentioned, the FreeBSD kernel can run ELFv1 and ELFv2 binaries alongside each other. (but it's best to at least keep them isolated in different chroots, if only to prevent confusion regarding unloadable libraries, since we will not be using different ABI libdirs like we do for running 32 bit programs on a 64 bit install, and the runtime dynamic linker is not named after the ELF ABI, so will conflict if you try and mix ELFv1 and ELFv2 in the same sysroot.)

[bdragon28]

BTW, for reference, the detection code for LLVM that we will be running with the FreeBSD in-tree LLVM and asking to be applied to upstream LLVM once flag day happens is at https://reviews.freebsd.org/D20383

[q66]

I wrote some code for the target_abi cfg here, but haven't been able to test it or expand upon it yet (been busy with other things). I should be able to look into it in the next few days though.

[steveklabnik]

Triage: not aware of any updates here

[brad0]

This also applies to OpenBSD/powerpc64 which uses the ELF v2 ABI.

[q66]

didn't find time to look into this further, sorry - maybe i'll try again in the next few weeks as i have more free time

[cuviper]

See also https://github.com/rust-lang/rfcs/pull/2992