[NO-ISSUE] fix nix

[MangoIV]

Proposed changes

• update nix code
  • remove unnecessary custom overrides
  • refactoring (restructuring, separate out large nix files)
  • include linting and formatting
  • unbreak the build (tests, haddock, run)
  • focus on simplicity (please note in your review if the nix code is in some place hard to read, such that I can add some comments
• new nix workflow
  • check nix code
  • check build in nix
  • check formatting of nix and haskell code in nix ci
• overhaul of the documentation
  • include docs on how to build flora with nix
  • how to use nix to provision devShells
• update the dependencies of flora

Warning action has to be taken by a maintainer to add a cachix api key

Contributor checklist

• My PR is related to \ [NO-ISSUE]
• [x] I have read and understood the CONTRIBUTING guide
• [x] I have inserted my change and a link to this PR in the CHANGELOG

[MangoIV]

I’ll fix tests in nix, then add some docs probably

[jappeace]

is it part of CI? consider adding both the shell and nix build to ci so it keeps working, eg https://github.com/jappeace/haskell-template-project/blob/master/.github/workflows/nix.yaml#L29

hecate will just come complainin to us if it breaks, but if they really want nix to function then this is the only way.

[kenranunderscore]

I hope I can find some time over Easter to try it out!

So if nix build works and is part of CI, that's a good first step towards securing it won't silently break again. It would still mean the nix-built binaries use (potentially lots of) different package versions (and the shell would provide those) compared to the freeze file, right? I haven't worked with freeze files before and haven't googled yet, but I can't imagine a good mechanism to have it really synchronized with the nix setup without making nix "lead" -- which would be undesirable imho, given that the non-nix, non-docker setup is the only one that was seemingly used lately :laughing: Can you think of any (maybe without bringing alternatives like haskell.nix into the equation)? (It would surely be "close enough" for most cases, though.)

[MangoIV]

the issue always is, if you generate the package set entirely from a freeze file, the caching will reduce and the IFD will increase, these are both really bad, and one of the reasons why haskell.nix is so slow.

[MangoIV]

I think this is functionally ready. Needs some docs and we're gtg

[MangoIV]

Nvm I just realised that running the test that nix works outside of the nix tests again is dumb. Will change that.

[MangoIV]

Would be happy about a review cc @kenranunderscore @jappeace

[kenranunderscore]

I can't check anything/much right now, will try later. Some questions/remarks I had without any checking, so take these with a grain of salt:

• I wonder how this works with souffle, or did you change the nixpkgs-unstable pin to use an older commit (or anything else)? In my local setup souffle couldn't be 2.3 or else I got compilation errors in Haskell/C++ iirc.
• I feel like the "open tasks" section should rather be tickets (as the first one already is). I'm also not sure if containers and deployment with Nix are something that's ever going to be used in the near future. Depends on how flora.pm is deployed right now I guess. I'm under the impression that @tchoutri is doing most of that on their own, without Nix, and with a different "version" of the app (due to using the freeze versions); whatever they use is the "canonical" version in my view.
• I don't know about the exact purpose of CHANGELOG.md and assumed it was only for changes in functionality. Someone else will know^^

[MangoIV]

I wonder how this works with souffle, or did you change the nixpkgs-unstable pin to use an older commit (or anything else)? In my local setup souffle couldn't be 2.3 or else I got compilation errors in Haskell/C++ iirc.

I changed the deps so they work with the new souffle.

I feel like the "open tasks" section should rather be tickets (as the first one already is). I'm also not sure if containers and deployment with Nix are something that's ever going to be used in the near future. Depends on how flora.pm is deployed right now I guess. I'm under the impression that @tchoutri is doing most of that on their own, without Nix, and with a different "version" of the app (due to using the freeze versions); whatever they use is the "canonical" version in my view.

that's fair, I can remove them if other people agree, I think I'm waiting on Hecate's view on this

[MangoIV]

the nix workflow lasts almost two hours because no caching :sweat:

[MangoIV]

good news everyone, I think it's possible to have cachix caching :3

[jappeace]

yeah, the one I linked has cachix :sweat_smile: you can use local cache as well

[MangoIV]

you can use local cache as well

you mean caching the nix store in between builds? afaik this doesn't work or at least didn't for a long time

[MangoIV]

work flow run times: 1h 50 min -> 10 min

[MangoIV]

I will convert to draft for the minute because i'm having some issues forgot to update the freeze file after updating to souffle 3.5.1

[MangoIV]

the shellHook also makes sure that the datalog always gets regenerated, which is also quite cute I think

[jappeace]

awesome!

[kenranunderscore]

@MangoIV Sorry I didn't get to really test it, but it seems like the others did so. Great work, I'll surely try it out soon :)