search

andir / npins

Nix dependency pinning. Very similar to Niv but has a few features that I personally wanted.
European Union Public License 1.2
164 stars 12 forks source link

A rough import-flake sub command #70

Closed stuebinm closed 2 months ago

stuebinm commented 3 months ago

i.e. a thing analogous to npins import-niv:

> npins import-flake
[INFO ] Note that all the imported entries will be updated so they won't necessarily point to the same commits as before!
Importing nixos-mailserver
Importing nixpkgs
[INFO ] skipping indirect input nixpkgs_2

I wrote it since, recently, I have once again become somewhat annoyed at my own usage of nix flakes (among many other things). So here I am. Thanks for all your work! (I used niv a couple years ago, and it feels nice to have something like it again — although of course niv also still exists).

While moving my own configs, I decided I didn't feel like moving all my old inputs across by hand, so I wrote this instead.

Unfortunately, this does mean it has some limitations. While I've modelled it on the existing niv importer, it covers not much more than exactly what I needed. So it is not very complete; it supports only git imports (github, gitlab, generic git), but flakes can also have many other types of input. On the other hand, as this is already (I think?) the entire common subset of input types between npins and flakes, this might be the best we can do?

Like the niv importer, there's no effort to keep the pins/hashes (although they are read & it should probably be possible to keep them — but I decided I didn't mind the dependency bump, so I've not looked into this further).

It is probably also generally very brittle — as far as I could see, there is not much documentation on the actual format used in flake lock files (the closest in the manual are some examples, which further appear to be out of date, or at least noticably differ from how my flake.locks look like). So while it has worked fine for me on the couple flakes I tried it on so far, it might very well break on another.

Finally, probably the biggest problem is how to document this, because while automatically transforming the lockfile helps a fair bit, a transition from a nix flakes-based config to an npins-versioned one is still somewhat bumpy (I now have various little hacks and shims in my nix expressions so I can still use dependencies which do not expose a non-flaky interface), and I don't see how it can really be automated more than this. Particularly, the importer skips indirect dependencies (it could also keep them — but then what would it do with them?).

stuebinm commented 2 months ago

I've added a simple test checking that a git repository is imported correctly, and that afterwards the npins sources.json contains the correct branch.

I tried doing the same with a github-API-based import, but this fails on the Nix side: while flake refs using the github: scheme can take a host parameter which we could use for the test API mock, Nix will "validate" that host against a somewhat naive regex that rejects the : in localhost:9000, so generating a fitting flake.lock for the test seems a little less straightforward.

andir commented 2 months ago

I tried doing the same with a github-API-based import, but this fails on the Nix side: while flake refs using the github: scheme can take a host parameter which we could use for the test API mock, Nix will "validate" that host against a somewhat naive regex that rejects the : in localhost:9000, so generating a fitting flake.lock for the test seems a little less straightforward.

I think this is fine. We don't have to produce a perfect lock file. All I really care about here is that the import works for a basic case, and we get a failure if it doesn't.