Explore magic-nix

[ch1bo]

Another trial of magic-nix which uses the github internal cache as a binary cache for nix.

Magic nix does not offer out-of-the-box any public cache interaction and we would need to make this work together with cachix. In another exploration, @locallycompact encountered problems by having both enabled: https://github.com/DeterminateSystems/magic-nix-cache/issues/78

However, before tackling this I wondered what the build time improvement would be. This PR contains commits and links to the findings:

• Plan: Getting a first single package built and rebuilt with both magic-nix or cachix.
• Magic-nix:
  • Using accept-flake-config to seed magic cache from cachix.
  • Rerun of hydra-node build with primed cache: https://github.com/cardano-scaling/hydra/actions/runs/10418269282/job/28854308115 -> build step only: 1m04s, overall 2m49s (still uploading)
  • Most time is spent on evaluating the derivation (IFD)
  • Post action uploads are rate limited - does a typical rebuild result in enough uploads?
  • Seems like magix-nix post action is re-uploading similar store paths on each rerun -> overall build time counts: 3m14s Are our builds non-deterministic due to IFD?
  • Rebuild hydra-node (changed version number) with magic nix: https://github.com/cardano-scaling/hydra/actions/runs/10418479282 -> 4m45s and no pushed paths pushed?
  • Rerun of the same action also resulted in a rebuild (indeed no paths were pushed) -> 4:19s, also no paths pushed again
• Cachix:
  • Switching over to using cachix-action https://github.com/cardano-scaling/hydra/actions/runs/10419403396 -> rebuild of hydra-node in 4m47s
  • Rerun of cachix build https://github.com/cardano-scaling/hydra/actions/runs/10419403396/job/28857732485 -> 1m28s, 1m50s
  • Dedicated rebuild of hydra-node with cachix https://github.com/cardano-scaling/hydra/actions/runs/10419717683/job/28858340501 -> 1m25s (accidental cache hit)
  • Now with a proper, new version for rebuild with cachix: https://github.com/cardano-scaling/hydra/actions/runs/10420234054/job/28859845924 -> 5m04s
  • Rerun after this https://github.com/cardano-scaling/hydra/actions/runs/10420234054/job/28860297642 -> 1m39s
• Conclusion:
  • Magic nix is slightly faster in fetching (from github cache than download from cachix), but also spends more time in pushing paths after builds
  • Also saw some paths pushed too often, or not at all (e.g. the deliberate rebuild of hydra-node was never cached)
  • It does not provide public access to the cached derivations
  • Not worth it

This write-up is also available on the logbook: https://github.com/cardano-scaling/hydra/wiki/Logbook-2024-H1#2024-08-16

---

• [ ] CHANGELOG updated or not needed
• [ ] Documentation updated or not needed
• [ ] Haddocks updated or not needed
• [ ] No new TODOs introduced or explained herafter