builtins.fetchTarball reproducibility issue (linux vs macos)

[DavHau]

Describe the bug Tarballs extracted on MacOS can lead to a different result than on linux. MacOS uses case insensitive filenames, therefore paths contained in the tarball can collide when extracted on macos. Nix silently ignores that and as a result the computed checksum differs on the two systems.

This violates the idea of nix builtins being platform independent.

It also impacts flake inputs.

Steps To Reproduce The following will evaluate under linux, but raise a checksum mismatch under MacOS

> nix repl
nix-repl> builtins.toString (builtins.fetchTarball {url = "https://github.com/commercialhaskell/all-cabal-hashes/tarball/hackage"; sha256 = "145fk6n0dvdhjxz9iyg08dbfvp3bcaqh49z9abb6b5h0jhbbyxyg";})

We also verified this the other way round. The checksum computed on macos via nix-prefetch-url --unpack will lead to an error when used on linux.

Expected behavior Not sure about this. The problem with that impurity is, that it stays undetected until evaluated on another platform.

Nix could always crash if a tarball contains paths which would collide under macos, even if extracted under linux. This would prevent users to write nix expressions that are not platform independent.

nix-env --version output nix-env (Nix) 2.11.1

[NobbZ]

In general this issue is known, as it has been stumbled upon in nixpkgs. There the issue usually got circumvented by using a FOD-hash that depends on pkgs.system, when it was confirmed that the result would have worked either way (eg. because it did not actually use the clashing files).

The alternative was to use a completely different source for mac and linux. Which also happens occasionally for this or other reasons.

Though with flakes rising, this becomes a problem with no way to circumvent! There is no way to specify alternate hashes for the same input.

It is also not easy to create the mac hash on linux and vice versa.

With limited knowledge about what would actually work on Mac, mixed up with a lot of hearsay…

If we were mounting the /nix as a case sensitive filesystem on macs and linuxes alike, wouldn't that solve our problems?

I've heard "native" Mac software would run into trouble with unconditionally enabling case sensitivity for Mac, though most if not all software reading the store should be made for "proper" linux anyway, and would therefore not necessarily break under case sensitive file systems.

Of course I am not even sure if that would be possible for individual mounts…

The other alternative would be to make the input mechanism aware of the problem and let it create both hashes by simulating both FS' behaviour and check for the correct hash on the respective platforms.

[edolstra]

Nix could always crash if a tarball contains paths which would collide under macos

It should only barf on macOS. I remember the Linux kernel source tree used to contain case collisions, so it would be unfortunate if we can't support such archives on Linux.

Alternatively, we could mangle clashing file names similar to what we do with NARs that have case collisions (see https://github.com/NixOS/nix/blob/62960f32915909a5104f2ca3a32b25fb3cfd34c7/src/libutil/archive.cc#L266-L274).

[toonn]

I'm not sure this problem can really be dealt with by Nix, other than adding case-sensit"ive file system without unicode normalization or file size limits (feel free to add other required properties like unix-style permissions ad nauseam)" as a requirement for running it.

The problem is more general than just Linux v. macOS, there are case-insensitive file systems for both and there are other problematic file system properties. Though this being the default does make it a common issue.

Automated mangling might side-step the issue of getting different hashes for the fetched source but will inevitably require patching source files for changed file names. I've used the mangling approach before (in a postFetch) for some test file names that happened to normalize differently on HFS+ but that did require also updating the file names in the testsuite code.

Ideally if an upstream source cares about compatibility across a wide variety of file systems, they can be made aware and fix it at the root. If an upstream is unsympathetic to the problem or it's impossible to resolve (an old release may not be alterable) then the problem is likely to require case-by-case fixes anyway.

Tooling that can notify of problems for the most common cases would be nice though. Something like a flag to nix-hash and the like that notifies of potential issues like case-collisions or NFC/NFD differences. Potentially also a flag that attempts to generate appropriate hashes for the most common scenario on each platform, for *-darwin it would handle both case and unicode normalization and spit out the appropriate hash, useful for cases where a package has multiple hashes for sources and they need to be bumped.

[abathur]

If we were mounting the /nix as a case sensitive filesystem on macs and linuxes alike, wouldn't that solve our problems?

As @toonn notes it wouldn't solve the problem, but it would minimize this specific sharp corner. When I worked on volume encryption for the macOS installer I intentionally left easy-to-fiddle envs so that people could experiment with changing the filesystem, but I don't think many people have reported back on their experiences.

The main (small) downsides, I think, are:

• It's not a quick fix; case-insensitive macOS systems will still make up a good share of the install base for some number of years (and the burden of discovering/fixing these incompatible sources will likely fall on a shrinking group of users.)
• These cases would start to accrue in private packages and nixpkgs again and increase the amount of pain we'd need to incur if we ever decided to move /nix (into /opt or elsewhere).

[aakropotkin]

Emitting a warning and allowing the user to deal with it as an edge case seems like a good solution to me.

I'd be more concerned about the edge cases that mangling would introduce obfuscating the issue and adding complexity.

[NobbZ]

What worries me most, is that because of this flakes can break for the system not used by the person who ran the last nix flake update.

And that there is no way around that.

The case that initally uncovered the issue has been that I was unable to build my blog using dream2nix for nodejs on a mac, as for some reason the haskell hashes are download from the transitive inputs, despite lazy fetching.

As I was not using haskell but node, I was able to mitigate the overal issue by using follows and swapping the input for nixpkgs and then my node build suceeded.

Doing tricks like this is not always possible, and in case for dream2nix and haskell that would mean, that one can not use it for haskell on mac at all. Or they use follows for a mac compatible input, in which case one was not able to use the result on linux.

And this is what concerns me. I don't think we will ever be able to find a solution that creates a "stable" hash for all supported systems, far too many edge cases.

But we need to find a way to selectively "override" the inputs on a per system basis, to be able to mitigate issues like this once awareness is risen.

[DavHau]

It should only barf on macOS. I remember the Linux kernel source tree used to contain case collisions, so it would be unfortunate if we can't support such archives on Linux.

@edolstra Why not make a flag ignoreCollision that allows the linux archive to be extracted. This would make the safe way the default, and the unsafe way an exception, that needs to be allowed intentionally. Similar to how nix flakes can allow impurities only by passing --impure.

I don't think warnings are any good, they will be ignored by some maintainers and will end up at the end users screen, eventually leading to confusion. It's best to force the maintainer to react to a potential problem immediately and make an intentional decision.