tazjin / nixery

Container registry which transparently builds images using the Nix package manager. Canonical repository is https://cs.tvl.fyi/depot/-/tree/tools/nixery
https://nixery.dev/
Apache License 2.0
1.8k stars 67 forks source link

underscore-prefixed packages can not be accessed #158

Closed orenmazor closed 1 year ago

orenmazor commented 1 year ago

howdy! I'm not the biggest nix user, but nixery has really gotten my attention.

we're testing it out to see if we can use it for transient images for our CI use cases.

but I've encountered an issue where I can't find a package that should exist. so I'm thinking I'm probably referencing it incorrectly:

I've looked at the sources file and it looks like the current instance of nixery is referencing 21.11, so I've adjusted my reference to the 1password-cli package accordingly. in the 22.11, the package naming appears to have changed slightly but the pname should still work?

➜ docker run -ti nixery.dev/shell/1password-cli/curl bash
Unable to find image 'nixery.dev/shell/1password-cli/curl:latest' locally
docker: Error response from daemon: manifest for nixery.dev/shell/1password-cli/curl:latest not found: manifest unknown: Could not find Nix packages: [1password-cli].
See 'docker run --help'.
zsh: exit 125   docker run -ti nixery.dev/shell/1password-cli/curl bash

what am I missing?

tazjin commented 1 year ago

Hey!

The current commit of nixery.dev is the one pinned as nixpkgs-unstable (which closely tracks nixpkgs' master branch) in this file: https://cs.tvl.fyi/depot/-/blob/third_party/sources/sources.json (barring deployment lag, as releases of nixery.dev are triggered manually).

I run the same commit locally and just checked, the package exists for me as _1password. Note the underscore, which is because of a restriction on identifiers in the Nix language.

This makes the whole thing a bit tricky, as the Docker registry protocol in fact forbids segments from starting with underscores, so this mismatch between the Docker/Nix references currently has no bridge between them. We have a similar problem because the Docker protocol forbids uppercase characters, which are also used in nixpkgs, and this is worked around in Nixery's Nix code.

I think what I'll have to do is add a similar workaround to find matching, underscore-prefixed package names if a package name starts with a number. I'll rename this issue and keep you posted.

tazjin commented 1 year ago

This is worked around in cl/8531, but that'll have to percolate through the review etc. first.

There's another problem here though: 1password is an unfree package which we're not allowed to redistribute:

nix-repl> _1password.meta.license.redistributable
false

That means even with this fix, nixery.dev won't be able to serve this package to you for legal reasons. You'd have to run a custom Nixery instance, in which you enable the distribution of unfree packages.

orenmazor commented 1 year ago

gotcha! thanks for clearing it up!

I work for 1Password and we're definitely looking into running our own instance so this is gonna come together nicely for us.

thanks again for digging into this!

tazjin commented 1 year ago

Cool!

The change is now merged here, if you build the latest commit it should work fine. It's going to take some more time to percolate through to nixery.dev, but I think in this case that won't make a difference so I'm going to close the issue. Please feel free to reopen or make a new one if you run into anything else :)