`post-build-hook: warning: unknown experimental feature 'fetch-tree'` and random CI failures

[dpc]

After switching to nix-fast-build I've noticed much more unexplainable CI failures. Oftentimes they look like random network issues, but recently I've spotted some really odd ones.

From our CI:

post-build-hook: warning: unknown experimental feature 'fetch-tree'
running post-build-hook '/home/runner/work/_temp/cachix-daemon-KNU4uw/post-build-hook.sh'...
post-build-hook: warning: unknown experimental feature 'fetch-tree'
copying path '/nix/store/vbh4481bzdv7pnqq8m3d0jwz27c0qx0x-clang-wrapper-16.0.6' from 'https://cache.nixos.org/'...
building '/nix/store/pg3ilwm16il5b1cbr4dx4cv0a2fmvjr9-llvm-config.drv'...
running post-build-hook '/home/runner/work/_temp/cachix-daemon-KNU4uw/post-build-hook.sh'...
post-build-hook: warning: unknown experimental feature 'fetch-tree'

It's unclear to me what could be causing it. cachix-action? nix-fast-build?

[Mic92]

nix-fast-build will bundle it's own nix interpreter in the build. Maybe it's older than the one your github action is expecting? You could test by doing an override.

[Mic92]

It's actually nix-eval-jobs, dictating the nix version here: https://github.com/Mic92/nix-fast-build/blob/4376b8a33b217ee2f78ba3dcff01a3e464d13a46/default.nix#L3 nix-eval-jobs links against nix, and it would be weird to have two different version involved in a build.

[dpc]

Hmmmm... I think I understand, but I'm not sure what's the conclusion. Also - that particular problem happened only once and it's now gone.

What's been plaguing us are flaky downloads:

copying path '/nix/store/mx1mwi6bfzjsblhw1m95qs1zwb0l382c-ndk-bundle-23.0.7344513-rc4' from 'https://fedimint.cachix.org/'...
Error: The operation was canceled.

for which changing the nix.conf settings for timeouts and staled timeouts doesn't seem to help. It's always the same path, relatively larger than other ones (Android NDK). No worries if you have no ideas - I can't even pin point the issue here, but I think it increased / start happening when we've switched our CI to nix-fast-build.

[Mic92]

Are you on a memory constraint device?

[dpc]

Are you on a memory constraint device?

No. There are Jetbuild Github Action runner boxes, so better than usual GH Action runner ones.

However I must report that after adding these two options:

      - uses: cachix/install-nix-action@v26
        with:
          nix_path: nixpkgs=channel:nixos-23.11
          extra_nix_config: |
            connect-timeout = 15
            stalled-download-timeout = 15

Number of hanged downloads decreased, though I think it didn't completely disappeared.

It's really unclear if it has nothing to do with nix-fast-build itself, so I guess I'm just going to close and post any updates if I ever have any. Maybe it will help someone googling this problem.