dkp pacman glibc 2.28 dependency

[CTCaer]

Any reason why latest dkp pacman is compiled with 2.28 as minimum requirement when it doesn't actually use any new api change? Previous one had 2.17 as requirement.

Because basically there are 2 breakages again:

1. glibc 2.28 (which is a hard blocking breakage because of system wide change needed)
2. zstd compression (which breaks all older dkp pacman and it's not ok when dkp-pacman is not updated on its own, or can be broken without notice if updated)

OS: linux/glibc 2.27/amd64

[WinterMute]

What distro are you using that's still on 2.27?

We use debian oldstable as a base, currently Buster. Usually because that has the oldest still maintained glibc. If your current distro has an older glibc it's probably time to update.

dkp-pacman package has stayed as .xz precisely to allow updating from earlier dkp-pacman builds which don't support zstd.

[CTCaer]

Hmm ok, so what I understand from this is that because of using debian as base distro for building, I'm forced to update the dev machine every 2 years just to get dkp updates or make any included tool to work, when it doesn't actually require such.

Also updating the distro, in this case, it's impossible, since it will break at least 100 projects in it. Half of them are impossible to even migrate (they have nothing to do with console dev). In real world, actually used linux based work/dev/build machines are not upgradable just like that.

Well anyway, thought to ask, since I didn't expect to get the somewhat modern linux response of 2-4yo distros are ancient.

[WinterMute]

Nobody said anything about forcing you to update every 2 years. We use Debian oldstable in an attempt to avoid precisely this kind of situation because it generally lags most of the distros people are using. Debian releases also have a 5 year lifecycle. I asked which distro you're using because I'm surprised that something is using an older glibc not because I think it's ancient.

We can look into what's causing the requirement for 2.28 & maybe adjust the pacman build.

[WinterMute]

Seeing as you asked so nicely pacman 6.0.2-6 will work with glibc 2.27

[CTCaer]

Thanks. That actually helps since I don't need to make the symbols weak or inject on every update.

The distros are rhel 7.9 (each minor number has a yearly release) with the highest possible glibc (and 5.10 kernel) that doesn't break everything, plus the basic libs and package stubs to make important debian tooling work. And also ubuntu 18.04.6 with the same generic and debian tooling. Both with ELTS (EoES/EoL of 2024 and 2028 respectively). If you need to stat needs based on popularity, Ubuntu 18 / Linux Mint 19 are probably enough. The former offers free ELTS and the latter is normally the classic one used in academia if linux is chosen and no one bothers to upgrade it timely.

The issue with 2.28 as a base is mostly a completely unnecessary breakage of the api compatibility. If I remember, no newer one had such an issue; an app will lose backwards compat only if it uses newer apis. Someone thought it's fine to suddenly force fcntl as fcntl64 (#define fcntl fcntl64). Which makes it even worse, because such practice can easily risk breaking the first rule of glibc, aka maintain backwards compatibility (e.g. app linked on glibc 0.1 must work on 2.37 lib).

[oreo639]

an app will lose backwards compat only if it uses newer apis.

I think you are mixing up backward compatibility and forward compatibility, glibc doesn't really support forward compatibility (when trying to run a binary compiled with a newer version of glibc on an older version of glibc).

The fnctl64 change was made here: https://github.com/bminor/glibc/commit/06ab719d30b01da401150068054d3b8ea93dd12f

Afaict the change was made to fix issues with how F_OFD_* was handled in the non-64 variant (since right before the change above the non-64 variant would just call the 64-bit variant without any special handling, whereas now it adds extra checks when handling the old flags, although it still just calls fnctl64 for the new 64-bit flags).

[CTCaer]

I'm not mixing anything. Just because you compiled the same app with a newer glibc, it doesn't mean that it now needs that one to work. It doesn't work like that. Symbols are pulled per usage and that's what expects a certain version. I already stated it was about api. So let's not play with words, when these technically used ones have a clear meaning, because what follows bellow shouldn't be the expected response if common sense exists.

If your app had no changes and it was compiled on a newer lib revision, and said lib's design is like glibc's, and now the same function call does not work on old, it's an api breakage. Plain and simple. Glibc clearly states every symbol expected version and only that is pulled during linking. And that makes the api breakage a broken backwards compat. If expected lower version compat is not there it's an api backwards compat breakage.

When you have GLIBC_2.2.5 on fcntl stated you expect your app, if it calls that function, to work on glibc 2.2.5. It's so simple as that. Not suddenly have it pull a GLIBC_2.28 fcntl64 hard dependency. Again assuming no other higher revisioned symbol was pulled, because that would make it fall into the stated "not support for library forwards compat".

No matter if the change was good or bad, needed or not, correct or false, it's still an api breakage and in that case it's a bad one because it's not clearly communicated and can't be avoided when possible.

Api breakages must be communicated and the best way to do that is to keep that in your list or at least in some changelog. Even on latest glibc, fcntl is stated as 2.2.5 dependant. Where it should have been 2.28 if __USE_FILE_OFFSET64 is defined. And imo, it should have allowed a compile time flag to disallow that behavior, in case developer doesn't use the affected flags and wants to keep the lowest possible compat with glibc.

Not even the shortlog states that:

* The fcntl function now have a Long File Support variant named fcntl64.  It
  is added to fix some Linux Open File Description (OFD) locks usage on non
  LFS mode.  As for others *64 functions, fcntl64 semantics are analogous with
  fcntl and LFS support is handled transparently.  Also for Linux, the OFD
  locks act as a cancellation entrypoint.

That means a new function was added. Use the new one if you want LFS support. Not a "A new function was added to add LFS support. Normal one will also use the new one if LFS is enabled". Worse even, it's phrased in such way that it looks like glibc handles that internally and not that LFS support constitutes that as a clear cut api breakage. So that function, till today, is still communicated wrongly as backwards compatible.

[oreo639]

Just because you compiled the same app with a newer glibc, it doesn't mean that it now needs that one to work.

That is why glibc has symbol versioning, so it can bump the symbol version and introduce ABI breaks without affecting currently existing applications (at the time of the ABI break). You can look at readelf -Ws '/usr/lib/libc.so.6' | grep '@' | sed 's/.* //' | sort for other examples. (i.e. dl*(), clock_*(), mtx_*(), pthread_*(), etc have all had their symbol versions bumped)

There are scripts online to generate headers to force linking to older symbols and that can probably be extended to fcntl(), but idk how well they work.

To be clear, I am not disagreeing with you that these kinds of breaks should be mentioned on the wiki or changelog, as most other libraries don't work like this.

[WinterMute]

This stackoverflow thread offers some insight into this issue. Personally I don't think this is reasonable either - Debian buster could easily build binaries that would still work on Ubuntu 18.04 if they didn't forcibly replace fcntl with fcntl64.