pin nixpkgs for bootstrap and provide a cache

input-output-hk / haskell.nix

Alternative Haskell Infrastructure for Nixpkgs

https://input-output-hk.github.io/haskell.nix

Apache License 2.0

556 stars 240 forks source link

pin nixpkgs for bootstrap and provide a cache #549

Closed domenkozar closed 1 year ago

domenkozar commented 4 years ago

I think all compilers should be just pinned to a nixpkgs revision by default and put on cachix.

This would avoid a lot of headaches that people are having with getting started and having to wait. It would mimic the experience with stack/cabal.

purefn commented 4 years ago

I think it would also be necessary to pin the hackage.nix and maybe stackage.nix used for nix-tools. I often have them pinned to newer version than the version of haskell.nix I'm using because I need newer packages from Hackage, but am not ready to bump haskell.nix itself.

When I do, I'm always surprised I have to build not just nix-tools, but often the whole toolchain up to nix-tools as well. It feels like nix-tools should always be able to be pulled from the cache, regardless of what nixpkgs or hackage.nix I'm using for my project.

angerman commented 4 years ago

don't we already do this? We've got all the build tools materialized now, and build for 19.09 and 20.03, as such all compilers for those two nixpkgs revisions should be pre-built? What am I missing?

domenkozar commented 4 years ago

@angerman since haskell.nix is used via overlays, people have issues finding the correct nixpkgs. It would be better if tooling nixpkgs would be pinned by default and came with a cache.

Then there could be a boolean option in module system to turn that off.

angerman commented 4 years ago

@domenkozar have you seen #514?

michaelpj commented 4 years ago

I think there's a general issue of approach here for external Nix repositories that contain a substantial amount of packages. Do you:

Use a user-provided nixpkgs, ensuring that everything will be in sync with the user's other packages, but causing more rebuilds; or
Use a pinned nixpkgs (at least for binary outputs), ensuring you get cache hits but risking that things will be out of sync.

I find the idea of not using the users nixpkgs a little scary. It seems like it could be a source of subtle unpleasant bugs: even if we just use it for e.g. GHC binaries, what about if the GHC we build is linked against a different version of a C library to a haskell program that it's building? Could that go wrong? I don't know, but it's a bit scary.

That said, nix-tools is a sufficiently simple input-output program that I think we could probably get away with building that against a fixed nixpkgs. Which would solve some of the pain points, since at least the eval-time dependencies would be cached!

domenkozar commented 4 years ago

I think nix-tools and GHC could be pinned by default. I understand that's scary, but it's no different to nixpkgs experience. It's only needed to mention that GHC doesn't come from nixpkgs but rather haskell.nix is also a distribution.

Rizary commented 3 years ago

I use haskellNix/nixpkgs-unstable but still cannot hit the cache. Is this normal? It's been a year and the issue is still open. Any problem that get in the way?

It's bad experience for me to try haskell.nix the first time, using a simple setup yet I have to compile ghc8107 and gcc again.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.