cachix / devenv

Fast, Declarative, Reproducible, and Composable Developer Environments
https://devenv.sh
Apache License 2.0
4.5k stars 339 forks source link

Feature Request: detect binary cache setup and fix error message or make silent #1604

Open xav-ie opened 4 days ago

xav-ie commented 4 days ago

Thank you for amazing project!!! 💞

I was hoping that the initial setup command of devenv init could somehow be improved to not error message at all?

After reading the message many times, I had honestly always thought I was doing something wrong:

❯ direnv allow
• Using Cachix: devenv
✖ You're not a trusted user of the Nix store. You have the following options:

a) Add yourself to the trusted-users list in /etc/nix/nix.conf for devenv to manage cache

trusted-users = root x

Restart nix-daemon with:

  $ sudo launchctl kickstart -k system/org.nixos.nix-daemon

b) Add binary caches to /etc/nix/nix.conf yourself:

extra-substituters = https://devenv.cachix.org
extra-trusted-public-keys = devenv.cachix.org-1:w1cLUi8dv3hnoSPGAuibQv+f9TZLr6cv/Hm9XgU50cw=

And disable automatic cache configuration in `devenv.nix`:

{
    cachix.enable = false;
}

Or maybe the documentation message can be enahanced to something like:

And now you may disable automatic cache configuration in `devenv.nix`:

{
    cachix.enable = false;
}
...since global caching is in effect. If you already have this configured, please just disable.

Or even just indentinting all of option a and option b?

It took reading https://github.com/cachix/devenv/issues/1527#issuecomment-2420609069 for me to finally realize I was not doing anything wrong, the tool does not seem to detect that you are setup properly and requires users manually disable cachix.enable as part of option b. The cachix.enable looks like it applies to both to me.

Is this trying to suggest I should set up a trusted user as my own user? Is that generally recommended? This error would never happen again if I did that, I think... However, I would rather it detect if I am set up properly with option b and disable cachix for me.

Next, I am guessing there is performance implication for having to manually do cachix.enable = false manually? In that case, could the documentation update be considered? I and others have gotten confused by it, and at a glance, a red ✖ in my console just means I did something wrong and have always thought I was setting up my binary cache incorrectly and the cachix.enable part was some final step for both a and b.

Finally, maybe the tool could parse nix.conf and check if it substituters are set up correctly and automatically disable the cachix.enable setting? Saying cachix.enable = false might be best left up to a tool that can detect if you did setup your cache substitution properly. Just disabling cachix does not mean you did the first part of option b correct.

sandydoo commented 1 day ago

Or even just indentinting all of option a and option b?

Sure, I can see how this could be confusing.

Is this trying to suggest I should set up a trusted user as my own user? Is that generally recommended?

Well, yes. That's the first option and it's the simplest. There are security implications to being a trusted-user, but IMHO, most users would be better off adding themselves to the list instead of dealing with the restrictions imposed by not being a truster user.

Next, I am guessing there is performance implication for having to manually do cachix.enable = false manually?

No? What sort of implications?

Finally, maybe the tool could parse nix.conf and check if it substituters are set up correctly and automatically disable the cachix.enable setting? Saying cachix.enable = false might be best left up to a tool that can detect if you did setup your cache substitution properly. Just disabling cachix does not mean you did the first part of option b correct.

Yes, devenv should query Nix for the configured substituters/keys. And because it currently doesn't, that's why the message instructs you to set cachix.enable = false; in option b.

sandydoo commented 1 day ago

This should improve things by actually checking if option b is set up correctly: https://github.com/cachix/devenv/pull/1610/files

theurgi commented 1 day ago

I'm confused by this as well. Hopefully my user experience is helpful...

In my case, I want to set up a simple Python environment with the uv package manager:

Given that I want to use the cachix cache, I don't know why I would set cachix.enable = false;, it's unintuitive. But even when I do add cachix.enable = false; to my devenv.nix, I still see the warning.

In all cases wherenixpkgs-python is declared as an input in my devenv.yaml, I see the warning but the shell builds. However, the process of pulling in dependencies differs greatly, and I'm not sure why.

The best case in terms of perceived work (i.e. the amount of time required to build the shell) is when my devenv.yaml is as follows:

inputs:
  nixpkgs:
    url: github:NixOS/nixpkgs/nixpkgs-unstable
  nixpkgs-python:
    url: github:cachix/nixpkgs-python
    inputs:
      nixpkgs:
        follows: nixpkgs

Removing the nixpkgs-unstable input results in an apparent scan of the entire Python universe. For fear of dying before it completes and/or consuming all my disk space, I abort the build.

Here's how I've configured NixOS:

{
  nix = {
    settings = {
      substituters = [
        "https://cache.nixos.org/"
        "https://nix-community.cachix.org"
        "https://devenv.cachix.org"
        "https://nixpkgs-python.cachix.org"
      ];

      trusted-public-keys = [
        "nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs="
        "devenv.cachix.org-1:w1cLUi8dv3hnoSPGAuibQv+f9TZLr6cv/Hm9XgU50cw="
        "nixpkgs-python.cachix.org-1:hxjI7pFxTyuTHn2NkvWCrAUcNZLNS3ZAvfYNuYifcEU="
      ];
    };
  };
}

I confirm the result:

❯ nix config show | rg substituters  
substituters = https://cache.nixos.org/ https://nix-community.cachix.org https://devenv.cachix.org https://nixpkgs-python.cachix.org https://cache.nixos.org/

❯ nix config show | rg trusted-public-keys
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs= devenv.cachix.org-1:w1cLUi8dv3hnoSPGAuibQv+f9TZLr6cv/Hm9XgU50cw= nixpkgs-python.cachix.org-1:hxjI7pFxTyuTHn2NkvWCrAUcNZLNS3ZAvfYNuYifcEU=
xav-ie commented 1 day ago

Wow, geez, thank you so much! I was really not expecting any type of solution so quickly. I think the changes made in the #1610 pr solve everything in the issue.

No? What sort of implications?

I guess I meant more like maybe there is reason user's must manually do this and devenv cannot "know" this. I thought it might slow things down to go into shell each time and have to check and this is why user's must choose this setting on their own. But that is not really on topic anymore :).

Also @theurgi,

Given that I want to use the cachix cache, I don't know why I would set cachix.enable = false;, it's unintuitive.

I think the PR wording changes make this setting more clear and solves that part.

But even when I do add cachix.enable = false; to my devenv.nix, I still see the warning.

I think the PR will fix this :)))). ➡️ "This PR queries Nix to figure out which caches and keys are missing. If the user is not trusted, but all the caches are already configured, no warning is shown."

Removing the nixpkgs-unstable input results in an apparent scan of the entire Python universe. For fear of dying before it completes and/or consuming all my disk space, I abort the build.

This seems like a separate issue about more advanced caching not being picked up somehow or maybe not configured correctly. I don't own this project or anything, but maybe it might be better to make that separate issue? I am mainly eager to have the core of my issues fixed being marked as resolved because it means I get this fix sooner 😄. Ultimately, I am out of my depth here and up to @sandydoo and you to determine best course of action here.

theurgi commented 1 day ago

@xav-ie Thanks for clarifying what the PR specifically addresses, I had missed that. You are right, my other issue is likely tangential and there's no need to suspend the resolution of this issue on my behalf.