The which_freebsd detection is cross-compilation unfriendly

[nagisa]

The detection of freebsd version invokes a freebsd-version executable:

https://github.com/rust-lang/libc/blob/5f7c6c1ef93ff4fcfa3d767aac93c2b261c8f140/build.rs#L122-L123

however that only meaningfully works when building on a freebsd host. When building on any other host and targeting a freebsd, libc will always default to targeting a freebsd11.0:

https://github.com/rust-lang/libc/blob/5f7c6c1ef93ff4fcfa3d767aac93c2b261c8f140/build.rs#L26-L34

Thus making it difficult to e.g.

• Cross-compile a library or binary that relies on freebsd12.0 or freebsd13.0 APIs;
• CI test other libraries that rely on freebsd12.0 or freebsd13.0 APIs.

Note that adding a freebsd-version executable to the path is not a very viable solution – some builds running in parallel may target other versions of freebsd too.

[JohnTitor]

Hm any suggestions here? I'm not familiar with FreeBSD, would it help that we could override the FreeBSD version by user flag?

[nagisa]

I recently just realised that this detection logic only works in CI anyway (if libc_ci) so even when compiling on a freebsd host you will only ever get freebsd11 APIs at most.

This kind of makes the freebsd12 and freebsd13 modules dead currently.

(I don't really have any good suggestions, on how to approach this, alas)

[nagisa]

Related: https://github.com/rust-lang/rfcs/pull/3036

[raphaelcohn]

The problem of versioning of the underlying libc is a hard problem that probably needs addressing via the compilation target 'tuple' - not that I'm a fan of these (we copied this very broken concept from GNU's autotools build system cruft - have you ever seen how hard it is for them to maintain their config shell scripts). This issue is going to come up time and again as Rust matures. A recent minor example was the transition to 64-bit time with musl 1.20.

More insidiously, it also creates hard-to-resolve 'not on my machine' bugs. For example, one might want a new function wrapper from, say, libc 0.2.95, but that binds system libc version Z - but we have only version Y. This creates a false promise only resolved when eventually linking (cargo build). Throw in the version of the OS, too, and it gets really hard. FreeBSD at least version the OS and the libc in lock-step; Linux, on the other hand... manages to make a right mess of what's a Linux API and what's a libc API, and the work the author of musl has had to over the years to get it to compile when the two don't agree has been extreme.

@nagisa The only approach that is ever going to work is a complete overhaul of libc which breaks everything downstream - libc 2 - and splits it out not just by target or OS, but maintains separate outputs by OS + OS version + OS libc (musl, glibc, etc) + OS libc version + static/dynamic. The later matters because certain functions (eg dynamic loading of .so) doesn't exist in statically linked libcs. Every function should have an attribute, comment or somesuch detailing the version of the libc it was introduced in, along with conditional compilation flags. Additionally, the current approach of lots of little contributions adding to libc has made the present situation worse. Personally, I' d then organise all the source so each file.rs maps 1:1 with the corresponding libc header - even where the headers are just redirects. I've had great success with this in my own projects, as it makes tracking what's changed a lot easier.

[bdragon28]

Note that for powerpc64 FreeBSD, we (the FreeBSD powerpc64 kernel/base team) changed ABIs entirely between 12 and 13 (from ELFv1 to ELFv2) so the lack of rust tracking the FreeBSD version currently prevents building for both without having mutually exclusive patches, as "powerpc64-unknown-freebsd" can mean vastly different things depending on what version is being assumed.

So in this case, it's not even possible to have things interoperate, they must differentiate on version. This is somewhat independent of the issue of libc symbols problem though.

(Additionally, the 32 bit powerpc platforms in FreeBSD switched from using bss-plt to using secure-plt in 13, which means clang et al will not generate correct code unless it is passed the version as part of the target. I've been banging my head against this while trying to port rust to powerpc-unknown-freebsd over the past week.)

Note also that the powerpc64 kernel knows how to activate both ELFv1 and ELFv2 images, so this mainly affects calling conventions for dynamic libraries (static doesn't matter since the kernel will enter the image with whatever the correct ABI is as per the e_flags value in the program header.)

[bdragon28]

Another thing that people coming from a Linux background might not be aware of is that the syscall API the FreeBSD kernel presents to a program can change slightly depending on the NT_FREEBSD_ABI_TAG ELF note on the main application. The system compiler will emit binaries with this set to the userland ABI tag (i.e. the value of __FreeBSD_version from /usr/include/sys/param.h at the time of compilation.)

(This also means that the dynamic linker /libexec/ld-elf.so.1 is required to adapt to whatever ABI version the program being dynamicly linked was tagged as. This can get interesting at times.)

As an aside, this is the reason that the BSDs generally require rebooting into a fresh kernel before installing an updated world when tracking the development branch -- the kernel needs to be new enough to understand the ABI revision that the userland wants to use.

The kernel knows how to speak old versions up to the limit of the oldest COMPAT_FREEBSDXXX options that were compiled into it. GENERIC kernels for a platform generally contain the ones going back to the origin of the platform so they can interact with old binaries.