error: input compression not recognized (private cache using zstd compression)

[andrewhamon]

Describe the bug

When I run nix build, I repeatedly get an error about compression when trying to substitute from a private cache.

error: input compression not recognized
error: some substitutes for the outputs of derivation '/nix/store/l59dvqj9lbm2n5agfx19hxbm2qlq4w0g-pnpm-9.0.6.tgz.drv' failed (usually happens due to networking issues); try '--fallback' to build derivation from source

The pnpm package in question is our own package that has this as its src:

  src = pkgs.fetchurl {
    url = "https://registry.npmjs.org/pnpm/-/pnpm-${version}.tgz";
    sha256 = "sha256-BiTjDv+GbN6zY7FQYb23/ZQlsXvBu0LCL19O/eoh9rM=";
  };

Steps To Reproduce

1. build a derivation that uses that src
2. push to a cache using zstd compression
3. clear your local nix store (rm result && nix-collect-garbage -d)
4. build again using your cache
5. see the error (can only fix by adding --fallback to the build)

Expected behavior

Cache substitutes correctly, or gives better advice on how to rectify the error.

nix-env --version output

nix-env (Nix) 2.23.3

Additional context

I only hit this when using zstd compression on the cache. The cache is a a gcs bucket (but using a s3:// cache URI).

Priorities

Add :+1: to issues you find important.

[andrewhamon]

I don't get this issue if I switch from compression=zstd to compression=gzip and overwrite the cache. So I think it is an issue specific to using zstd with a cache.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-08-12-nix-team-meeting-minutes-168/50561/1

[roberth]

Discussed during aforementioned Nix maintainers meeting:

We have a test for zstd compression for a binary cache

If this were HTTP, I would ask the following. S3 should be analogous. What does curl http(s)://your-cache.local/l59dvqj9lbm2n5agfx19hxbm2qlq4w0g.narinfo return? It should provide you with a nar.zstd filename. If you download that, is it in the correct file format?

Possibly related: https://github.com/NixOS/nix/pull/7712

[Mic92]

Possibly related: #7712

My PR was about http transport encoding (this is transparent to nix as curl handles it i.e. it should not appear in narinfo either), but the person here uses zstd compression on the content itself.

[andrewhamon]

If I find some time, I'll see if I can reproduce using minio. Unfortunately, my main specimen was serving production use cases and I already did the cache surgery, so I can't easily grab that info you are requesting.