Nix build is unstable; Should we keep it?

[Anton-Latukha]

Recently master & so all PRs started to: https://github.com/haskell/haskell-language-server/runs/4509561080?check_suite_focus=true

> nix build --print-build-logs
...
ghcide> Setup: Encountered missing or private dependencies:
ghcide> hie-bios >=0.8 && <0.9.0
...
error: 1 dependencies of derivation '/nix/store/ckcngbdahcmykc05rljj0cpkqvjicslb-haskell-language-server-1.5.1.0.drv' failed to build
Error: Process completed with exit code 1.

[jneira]

ouch, the nix build seems to be fragile lately, do we know if some pr broke it? maybe we should trigger the nix build in all prs and not only in master?

//cc @maralorn @michaelpj

[Anton-Latukha]

Yep, it is: https://github.com/haskell/haskell-language-server/pull/2470/files it raised the hie-bios bound to 0.8 & Nixpkgs current & that is now pinned use 0.7.1.

Nixpkgs (especially sure of haskellPackages) has a different versioning solving system (no versioning solving system) than Cabal & Stack, that is why these can't be avoided.

The downside is - putting checks to the PRs would now suddenly put on the unaware Haskell code writing devs a sudden requirement of Nix code maintenance. & some part of Nix loving Haskell devs should already use Nix dev env - so those PRs then should already be Nix-aware. & people which do not know Nix - would not be able to solve Nix things without maintainer support.

& then the solution starts from that position. As I see it - both variants under evaluation are the same value but with different side effects. Which is decided - is not for me to decide.

[michaelpj]

Well, or it could run on changes that touch cabal files only.

Honestly, I'm not convinced that the nix build as it stands is fit for purpose. HLS is a fast-moving project that frequently uses bleeding-edge versions of libraries that have just been released to Hackage. The nixpkgs Haskell setup is based on infrequently updated Hackage snapshots, and is more suited for building stable software.

As it is, to fix things like this would require some tedious patching in of the new versions of libraries every time someone tries to use a newly released library. Not a good UX for contributors.

(I don't even use the fancy Nix setup to work on HLS! I just work in a nix-shell with ghc and cabal and it's fine.)

[Anton-Latukha]

I agree that for HLS it is challenging to maintain Nix dev env. I agree that Nix code does not suite the bleeding-edge development, especially as close & as fast to the core ecosystem as HLS should. It basically would/requires constant repinings & overloading of the packages.

Nix is good for deployments of releases & for the use of projects locally. But development in it - means keeping to the Nixpkgs/flake versions & need to overload & then remove or reoverload everything additional. It is not that frequent for most projects, but still. I'd wished Nix packaging had the ability to have bound constraints mode for development purposes - that would've solved all this almost fully.

I did not create & I not maintained the Nix setup - so not me to decide about it.

Seems a simple straw pool can shine some data on this situation.

[jneira]

The downside is - putting checks to the PRs would now suddenly put on the unaware Haskell code writing devs a sudden requirement of Nix code maintenance. & some part of Nix loving Haskell devs should already use Nix dev env - so those PRs then should already be Nix-aware. & people which do not know Nix - would not be able to solve Nix things without maintainer support.

totally agree

[pepeiborra]

Honestly, I'm not convinced that the nix build as it stands is fit for purpose. HLS is a fast-moving project that frequently uses bleeding-edge versions of libraries that have just been released to Hackage. The nixpkgs Haskell setup is based on infrequently updated Hackage snapshots, and is more suited for building stable software.

Completely agreed. The previous shell.nix descriptor was designed with this in mind, I think the upgrade to Flakes has proven to be a step in the wrong direction.

[michaelpj]

cc @berberman

[Ailrun]

Will be fixed with this commit: https://github.com/haskell/haskell-language-server/pull/2482/commits/a6b5b32a927384e5f40af6fc1af55ba78c753b5b

[berberman]

nix build here is another distribution method, independent from cabal-install and stack, which means we have to duplicate the changes of the build config to nix.

Imagine that we want to update a dependency of ghcide, which was just released to Hackage. For cabal-install, the only thing we need to do is to update the package index. For stack, we need to add this new package to extra-deps of each stack descriptor. For nix build, first we need to distribute this new package to nix, and then put it into Haskell package set.

Given that the update frequency of Haskell packages in nixpkgs is not as fast as it used to be, the leaving maintenance cost is indeed high for us, and nix is totally optional for Haskell developers, so I have no objection to no longer ensure the build status of nix build.

[Ailrun]

This specific issue is now fixed, but it is maybe worthwhile to discuss whether we will actively maintain nix-based distributions or not. I will change the title of the issue.

[jneira]

Well i think we have to try to keep the nix build as it is a more and more popular way to build in the haskell community. Otoh the nix build usually is not triggered on pr's thanks to the cancel action, only in master, so it does not interfere with the development process. And we have several maintainers and contributors using nix who fixes the build pretty fast for now (btw thanks all!)

One path of action could be restore the previous shell.nix, as suggested by @pepeiborra. Would love to know the opinion on that of other maintainers on nix.

[fendor]

As some know, I am personally rather unhappy with the current build-tools HLS supports. Bumping a single library is an effort of changing at least 8 files (which used to be more, but we dropped some stack.yaml files), and I believe we should aim to reduce this number drastically. While I think shell.nix is great, the flake is not (firstly, because it is an experimental feature, secondly, because it duplicates again the information from cabal.project and stack.yaml) and is for newcomer HLS devs a bad experience, in my experience.

Therefore, I am completely +1 on removing nix-builds from our CI and not actively maintaining a nix-build that is more than zero maintenance work. (Edit: this opinion is the same regardless of whether the nix-builds are actually flakey or just temporarily flakey)

[Anton-Latukha]

Discussion & the theme of the report suddenly went a direction.

---

1. Nix dev env builds just can be made optional.

& so the people that use them maintain them. In that way Nix dev env can exist & be refactored without blocking the community processes. If Nix dev env falls to the wayside for several month at hand & there are no processes around it to fix it - means there is not enough use to maintain it.

---

One of the main problems I wanted to shine a light on:

afaik Nix users that are on NixOS still can not do cabal or stack development without Nix wrapper environment. cabal & stack just can not be used on NixOS. They need an existing nix shell wrapper.

That is the one thing that drove me from using NixOS for Haskell (& ironically HNix) development. Every Haskell project I needed/wanted to contribute - required me to make a working nix shell wrapper under widely different Hackage package constraints. & people frequently contribute to 15-20 projects on the regular. So needed to get out of situations of particularly NixOS distribution to be able to write Haskell code.

2. So, it is important to know/remember as a fact: if there is no Nix dev env (especially true for HLS case) - that means users of particularly NixOS are not able to develop for the project.

One of the situations was the HIE in NixOS back then. And I know how much effort Nixpkgs haskellPackages team puts to make things working & nice. Having a Nix dev env in the project essentially helps with the compatibility with the central distribution.

3. It is nice to collaborate with Nixpkgs team, so both projects have an easier time supporting HLS in the Nix ecosystem. Because HLS started to work in NIx/Nixpkgs thankfully to maralorn, project must be respecting of that also.

[fendor]

Discussion & the theme of the report suddenly went a direction.

When I looked at the title today, it contained already "should we keep it" :D

Nix dev env builds just can be made optional.

True, but I suspect this would yield rather bad situations for people expecting the build to work.

cabal & stack just can not be used on NixOS. They need an existing nix shell wrapper

I think they still do, and having a minimal shell.nix should mitigate that issue afaict.

So, it is important to know/remember as a fact: if there is no Nix dev env (especially true for HLS case) - that means users of particularly NixOS are not able to develop for the project.

If I am not mistaken, flakes are incompatible with nix 2.3, no? So, if people can't upgrade for some reason listed in https://discourse.nixos.org/t/nix-2-4-and-what-s-next they are currently unable to contribute.

Because HLS started to work in NIx/Nixpkgs thankfully to maralorn, project must be respecting of that also.

I don't quite understand how that is related right now. I totally agree, maralorn did some wonderful work for maintaining HLS in nixpkgs, but nixpkgs does not use the flake provided here?

[michaelpj]

nix build here is another distribution method

Right of course: at the moment we are doing two things with the Nix build:

1. It gives you a dev env setup
2. It gives you a way of actually installing an appropriate HLS as a user (distribution)

I don't know how many people are using 2, but if we go back to a minimal shell.nix we would keep 1 but lose 2.

This is where @Anton-Latukha 's point comes in: I think that indeed nixpkgs successfully distributes HLS now, which makes it less essential for us to provide a Nix-based distribution method. I don't know if the nixpkgs setup supports all the versions of GHC that we support, though?

afaik Nix users that are on NixOS still can not do cabal or stack development without Nix wrapper environment. cabal & stack just can not be used on NixOS. They need an existing nix shell wrapper.

I use NixOS and I develop HLS with a very simple nix-shell -p ghc cabal zlib invocation, and then cabal works just fine inside that. It can get hairy, but mostly it's fine.

[michaelpj]

Also, whether we want 1 or 2 affects whether we can let the build fail: if we only want 1, it's okay for it to be red and for the people who use it to keep it working in an ad-hoc way; if we want 2, then we cannot allow it to be red otherwise it will randomly not work for our users, which is a pretty bad UX.

[Anton-Latukha]

Well.

I would not continue to explode the dialog.

Considering that people fixed the Nix dev env & making Nix dev env process nonblocking in #2488 - enough. If someone would want to refactor it now - non-blocking allows to work on it while HLS develops.

That process is self-resolving.

---

The thread rapidly went out of scope of the initial report.

Thread can be summed-up & new report can be opened with pros/cons & have a more constructive start.

The reason for which this report was opened - is resolved & the opener of the report sees current results as a good approach to it.

---

P.S.

Right of course: at the moment we are doing two things with the Nix build:

1. It gives you a dev env setup
2. It gives you a way of actually installing an appropriate HLS as a user (distribution)

I don't know how many people are using 2, but if we go back to a minimal shell.nix we would keep 1 but lose 2.

This is where ... point comes in: I think that indeed nixpkgs successfully distributes HLS now, which makes it less essential for us to provide a Nix-based distribution method. I don't know if the nixpkgs setup supports all the versions of GHC that we support, though?

Considering it sound.

Nixpkgs generally supports 3 branches (officially) & more versions of GHC unofficially.

[maralorn]

I am a bit late to the party, but I just wanted to give my two cents.

1. I have no strong opinion if hls has a nix derivation in their repo. It is mainly orthogonal to the stuff we do in nixpkgs, although my hope is that maintaining hls in nixpkgs should ensure that dependencies (of released versions) build for your dev env. I sometimes use the projects nix file as inspiration for how to fix stuff in nixpkgs, though. (did that for ghc 9, e.g.) So it’s a bit helpful to me.

2. Given that the update frequency of Haskell packages in nixpkgs is not as fast as it used to be

I would like to dispute that claim. The update frequency was precisely once per week (every friday). Now it is a bit less regular, but one average more than once per week. Latency from hackage to nixpkgs- master might have increased slightly for releases on thursdays, but I don’t think it grew much on average for other days of the week.

Nevertheless, if you want a new version of your own release asap, you will need an override. (callHackageDirect is recommended). Also because after maybe 2 to 5 days on haskell-updates it takes another maybe 3 days until the next channel bump.

3. The claim that a project cannot be developed on nixos without a proper shell.nix is a bit strong. A dev can always throw a simple shell.nix with cabal, ghc and maybe zlib in and can then build most projects. I think the nix files are not mandatory for nixos users to develop on hls, but they are convenient.

(By the way maintaining hls on top of lts stackage in nixpkgs is pretty low effort this days now that the project has settled a bit. So the praise for the maintenance goes mainly back to you guys and your much improved hackage releases in the last few months.)

[pepeiborra]

I use the Nix shell setup to get access to a couple of tools: in addition to ghc and cabal, I rely on tracy to debug traces for performance, and opentelemetry-extra to convert eventlogs to Tracy. There's also the commit hooks that format the codebase.

In contrast, I don't use the nix build at all, and I think it's fair to say that it gets much less use. I believe the only benefit it provides is shorter bootstrapping times thanks to the Nix cache, in exchange for a fair amount of maintenance overhead.

In short: I am in favour of going back to the previous, more lightweight Nix setup, but I don't really have a strong opinion as long as we have a functioning Nix shell for contributors

[michaelpj]

I would like to dispute that claim. The update frequency was precisely once per week (every friday). Now it is a bit less regular, but one average more than once per week.

This is great... but frequently HLS is pushing people to do Hackage releases and then eagerly pulling them in a day or two after they go up, so...

Nevertheless, if you want a new version of your own release asap, you will need an override.

We would end up here a lot. Which sucks for most contributors.