libsodium 1.0.18.1 NuGet package is broken on some older platforms

[ektrah]

Note: This issue is about the libsodium NuGet package, not about libsodium itself.

My libsodium binding uses the libsodium NuGet package and fails its test cases on some platforms since NuGet package version 1.0.18.1:

In each case, the dynamic linker is not able to find or load libsodium.{dylib/so} from the nuget package. My guess would be that this is because libsodium.{dylib/so} has been compiled on a platform that is newer than the failing one.

@jedisct1 Feel free to assign this issue to me. I will investigate when I find some time. Many thanks for your support of the whole nuget stuff so far! 😃

cc @enclave-alistair

[ektrah]

Error message on macos10.13:

dyld: Symbol not found: ____chkstk_darwin
  Referenced from: /Users/distiller/project/tests/bin/Debug/net5.0/runtimes/osx-x64/native/libsodium.dylib (which was built for Mac OS X 10.15)
  Expected in: /usr/lib/libSystem.B.dylib

Error message on both centos7 and debian9:

Unable to load shared library 'libsodium' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable

[jedisct1]

Damn. Do we really have to support these old systems?

Your help would definitely be very welcome here!

[ektrah]

IMO, we'd ideally support all platforms supported by current .NET Core releases:

• .NET Core 2.1 - Supported OS versions
• .NET Core 3.1 - Supported OS versions
• .NET 5 - Supported OS versions

[udf2457]

* .NET Core 2.1 - Supported OS versions

I agree with @jedisct1 : "Damn. Do we really have to support these old systems?"

@ektrah As per the table on the page you linked to, EOL for .NET 2.1 is "August 21, 2021".

Given that date is a matter of weeks away, I suggest its a waste of time devoting a further second to .NET 2.1 support.

Better to freeze libsodium .NET 2.1 now to make the point that the project is not going to waste its time supporting things once they go obsolete.

Same opinion on ancient Linux and ancient OS X.

That's my 2c worth - similar to my opinion on those projects that insist on supporting Python 2 despite its demise !

We have to lead by example ! libsodium is a security library. Why should we be encouraging people who use ancient OS which no longer receive critical security patches ? No OS support = No libsodium support. Simple !

[ektrah]

This is not so much about supported .NET versions but about the OSs supported by .NET. Even the upcoming .NET 6 is supported by Microsoft on the "ancient" CentOS 7. I don’t think we’re encouraging people to use such OSs by doing this but rather enable people who have to use those for some reason to use the latest libsodium version. I’m more than happy to drop OSs that are no longer supported by .NET; for example, I think we can drop the support for macOS 10.13 now. So, indeed, no OS support (by a current .NET version) = no libsodium support. 🙂

[AJenbo]

I think we can drop the support for macOS 10.13 now

That would make me sad, my Mac is stuck on this version as Apple doesn't allow OS upgrades for older hardware.

[udf2457]

I think we can drop the support for macOS 10.13 now

That would make me sad, my Mac is stuck on this version as Apple doesn't allow OS upgrades for older hardware.

You must be running some pretty ancient hardware ? Current OS requirements:

• MacBook (2015 or later)
• MacBook Air (2013 or later)
• MacBook Pro (Late 2013 or later)
• Mac mini (2014 or later)
• iMac (2014 or later)
• iMac Pro (2017 or later)
• Mac Pro (2013 or later)

I think it is fair and reasonable to maintain a no OS support = no libsodium support policy.

Otherwise you just end up like the OpenBSD project with developers wasting time and resources supporting all sorts of ancient and obsolete platforms that hardly anyone uses "just because".

[AJenbo]

MacBook 2011, these things are expensive.

I think it is fair and reasonable to maintain a no OS support = no libsodium support policy.

It is, just this specific one makes me sad.

[enclave-alistair]

Only just seeing this (after running into precisely this problem on CentOS 7).

The problem with CentOS 7 is that the GLIBC version (2.17) is older than the version it was built under (2.25). It's a pretty old version, but CentOS 7 doesn't go EOL until 2024 I believe. Probably should support that. We might be be able to add an additional libsodium target specifically for the centos.7-x64 RID, built inside a CentOS docker container; I will try it. Same approach will probably work for debian9 as well. I will try a PR for it (hopefully this week).

Downside is that applications that consume the package will need to indicate they are targeting centos7 rather than the generic linux-x64 in order to get the working version.

macOS compatibilty is going to be more troublesome; there isn't a github runner for the older macOS versions, and if the OS itself is out of support, I'd have to say we avoid jumping through hoops build-wise to support it.

[udf2457]

MacBook 2011, these things are expensive.

I don't disagree with you about the cost, although you do pay for the build quality and reliability. I do still tend to end up replacing mine every 5 years, but then my use is quite intense and a 5-year cycle feels "right" for my needs.

Also, at the moment I wouldn't blame you for being like me and waiting a little bit longer for the M1 processor to be available across the MacBook range and not just a couple of models. :)

[udf2457]

CentOS 7 doesn't go EOL until 2024 I believe

Ah, CentOS 7 ... that old relic that refuses to die. Took them forever to finally bring out CentOS 8 but they're still clinging onto 7 like there's no tomorrow !

You are (unfortunatley) correct, "Maintenance Updates" until 2024-06-30 (https://wiki.centos.org/About/Product)

[enclave-alistair]

I think in general I agree with @ektrah, while .NET continues to support an OS version, so should the libsodium package (even if users have to jump through a hoop or two to consume the package).

[AJenbo]

I don't disagree with you about the cost, although you do pay for the build quality and reliability. I do still tend to end up replacing mine every 5 years, but then my use is quite intense and a 5-year cycle feels "right" for my needs.

I tend to have to replace the keyboard on them at a bout that rate.

Also, at the moment I wouldn't blame you for being like me and waiting a little bit longer for the M1 processor to be available across the MacBook range and not just a couple of models. :)

I appreciate the sentiment, but since I only use Mac for testing (maybe iOS porting, but not by choice) I tend to stick to Mac systems that has fallen out of favor from there previous owner :D

I also have a 2002 iMac, libsodium works just fine here since I installed Linux on it, so this is mostly an Apple support issue imo (would be amazing if they provided a VM image of macOS).

I would love to have an M1 system for testing, but considering that it's not able to dual-boot, even a low end Mac Mini would be prohibitively expensive for what I do.

P.s. sorry for all the off topic chit chat :)

[ektrah]

The problem with CentOS 7 is that the GLIBC version (2.17) is older than the version it was built under (2.25). It's a pretty old version, but CentOS 7 doesn't go EOL until 2024 I believe. Probably should support that. We might be be able to add an additional libsodium target specifically for the centos.7-x64 RID, built inside a CentOS docker container; I will try it. Same approach will probably work for debian9 as well. I will try a PR for it (hopefully this week).

Downside is that applications that consume the package will need to indicate they are targeting centos7 rather than the generic linux-x64 in order to get the working version.

I can’t test it right now, but I think the issue boils down to memcpy having changed between glibc versions. Before the libsodium 1.0.18.1 NuGet package, we had solved that problem by compiling libsodium on an ancient CentOS version. Maybe there’s a better way?

Overall, I don’t think it’s a problem to dynamically link libsodium against the older glibc for everyone, so having one binary for generic linux-x64 should be OK.

macOS compatibilty is going to be more troublesome; there isn't a github runner for the older macOS versions, and if the OS itself is out of support, I'd have to say we avoid jumping through hoops build-wise to support it.

A quick search revealed something called MACOSX_DEPLOYMENT_TARGET that maybe could be used to compile libsodium on a newer macOS version so that it runs on 10.13, but I didn’t have time to test that yet.

But, yeah, since macOS 10.13 is no longer supported by current .NET versions, I don’t think we should jump through too many hoops here.

[jedisct1]

Apple doesn't support macOS 10.13 any more either. They stopped providing security updates on December 2020.

So I don't think it is worth trying to support it.

This is sad for old hardware, but Linux and BSD are a better fit for old hardware. Still running an OS that is not receiving any security updates is not a good idea.

[enclave-alistair]

Raised a PR for CentOS 7 support. #1099.

[PJB3005]

So I see that this was fixed in tree, but no updated NuGet package has been published yet. Could this be done?

Also, what was the change between the 1.0.18.0 and 1.0.18.1 of the NuGet package anyways?

[ektrah]

@PJB3005 I can answer only the second question: The NuGet packages contain pre-compiled binaries for various platforms. The binaries in the 1.0.18.x packages are all built from the same sources (namely libsodium 1.0.18) but differ in how they are built and for which platforms.

Some of the binaries in the 1.0.18.1 package were built on newer platforms than in the 1.0.18.0 package, so they stopped working on older platforms. The binaries in the 1.0.18.2 package will be built on slightly older platforms again, so they will work on both newer and older platforms.

[PJB3005]

Well yeah, but what warranted recompiling them on new platforms in the first place? Improved performance due to newer compilers or something?

[ektrah]

@PJB3005 The platforms originally used to compile the binaries for the 1.0.18.0 package reached their end-of-life (EOL), so they were replaced with the latest and greatest for the 1.0.18.1 package. However, that broke some platforms that were a bit older but not yet at their EOL, so the binaries for the 1.0.18.2 package will be compiled using those older, not-yet-EOL'ed platforms.

[jedisct1]

1.0.18.2 uploaded to nuget :)