The way Hydra artifacts downloads works is that it runs "nix store cat", reads the whole contents into RAM, and then sends it as HTTP response. This has multiple issues for larger files (e.g. disk images):
Very long time to first byte, which could cause some clients to time out.
Very high RAM usage by the Hydra web workers.
Significant decrease in throughput, since the transfer time happens ~ 2x.
We noticed it a few weeks ago on #infra:nixos.org but then this got rediscovered again today on #hydra:nixos.org so I thought it might be worth filing a bug to actually track this.
https://github.com/NixOS/hydra/blob/c1bd50a80db45930cdfc1975903d1f76cca1053a/src/lib/Hydra/Controller/Build.pm#L239
The way Hydra artifacts downloads works is that it runs "nix store cat", reads the whole contents into RAM, and then sends it as HTTP response. This has multiple issues for larger files (e.g. disk images):
We noticed it a few weeks ago on #infra:nixos.org but then this got rediscovered again today on #hydra:nixos.org so I thought it might be worth filing a bug to actually track this.