LnL7 / nix-darwin

nix modules for darwin
MIT License
2.44k stars 408 forks source link

$PATH is broken for fish shell #122

Open grossbart opened 5 years ago

grossbart commented 5 years ago

I updated nix-darwin today and had to rollback to a version from October because my $PATH was broken (using fish shell).

Before (correct):

/Users/gape/.nix-profile/bin
/run/current-system/sw/bin
/nix/var/nix/profiles/default/bin
/usr/local/bin
/usr/bin
/bin
/usr/sbin
/sbin

After (broken):

/Users/gape/.local/bin
/Users/gape/.local/bin
/Users/gape/.nix-profile/bin
/usr/local/bin
/usr/bin
/bin
/usr/sbin
/sbin
/Users/gape/.nix-profile/bin
/run/current-system/sw/bin
/nix/var/nix/profiles/default/bin

The important part being that /run/currenty-system/sw/bin comes too late and the Nix provided binaries are not picked up correctly, resolving to /usr/bin/git for example. I'm also not sure where all this duplication is coming from …

I tried to get to the source of this error but was not able to figure it out exactly. What worked was to switch to an older generation and then to the newest generation again: in this case, the $PATH was set correctly. But when I exit fish shell and enter it again, the $PATH is broken.

It could be that this problem was introduced in https://github.com/LnL7/nix-darwin/commit/676ef103771aa3fc4b150290294b8ad5610d2750#diff-02a3bd02a5cdf5583b2e516a0e92d58a because the version of nix-darwin that worked for me was from 2018-10-17 and this is the major change that happened to the environment a few days later. But then maybe not because no one else has this problem?

clo4 commented 11 months ago

This is a quick recap of this issue for people without much experience with Nix internals (like me!), people that don't want to read the thread, or people that just want a solid solution.

What's the problem?

The script nixos-env-preinit.fish is executed on shell startup. You can find this at /etc/fish/nixos-env-preinit.fish on your system. This file executes another script that sets up environment variables, including the $PATH. The path is set up correctly here!

However, this happens before fish runs its own config scripts. Only when Fish is set as your login shell, it reads from /etc/paths and /etc/paths.d/*, sets that as the path, and then appends the pre-existing $PATH. Here's the code that does this. This is normally what you want, but in this case, it ends up putting your Nix directories at the end.

What paths exactly need to be moved?

Before Fish clobbers the path variable, the important directories in it are the /bin subdirectories of the values of environment.profiles. This is set, by default, by:

... but anything can add to this value. Using hardcoded paths is probably fine but isn't completely "correct" in the programmer-y ideology kind of way.

What's the fix?

The paths in environment.profiles need to be moved to the front of $PATH but behind $fish_user_paths, which is a magic fish variable that's always prepended to your path. (When you use the fish_add_path command, it prepends to $fish_user_paths universally by default. Universal means it modifies ~/.config/fish/fish_variables)

This module works correctly -- you can copy/paste it or save it as a file and add it to imports. I'm pretty new to Nix so this might not be super idiomatic, but this is about as correct as it can be I think.

{ lib, config, ... }:
{
  programs.fish.loginShellInit =
    let
      # This naive quoting is good enough in this case. There shouldn't be any
      # double quotes in the input string, and it needs to be double quoted in case
      # it contains a space (which is unlikely!)
      dquote = str: "\"" + str + "\"";

      makeBinPathList = map (path: path + "/bin");
    in ''
      fish_add_path --move --prepend --path ${lib.concatMapStringsSep " " dquote (makeBinPathList config.environment.profiles)}
      set fish_user_paths $fish_user_paths
    '';
}

You can probably get away without double quoting at all, it just makes me a little uncomfortable to think that a stray space could blow up my shell init 😅

The set fish_user_paths $fish_user_paths line is done for the side-effect of reordering $PATH, putting the user paths in the front.

(There are lots of valid ways of doing this -- another is to use string split " " $NIX_PROFILES)[-1..1]/bin, but this will always happen at runtime)

This is valid either in your home-manager config or nix-darwin config, both define programs.fish.loginShellInit, but as @yoav-lavi noted, you need to change config to osConfig when using it as a home-manager module.

A note about Fish's weird source order
Fish is kinda weird about what order to source things in. Currently, as of fish 3.6, fish sources user config **first** (~/.config/fish/conf.d/*), then system config (/etc/fish/conf.d/*), then vendor config, then user config.fish, and only *then* the system config.fish. This means using `programs.fish.loginShellInit` will execute **last**. Everything that gets executed before that will have the wrong path, but your interactive shell will be correct. This will be fixed in the future by sourcing config snippets (conf.d/*.fish) in order of name, regardless of the directory. So the *most* correct thing to do would be using something like ```nix environment.etc."fish/conf.d/00-nix.fish".text = let # ... in '' if status is-login # config snippet... end ''; ``` But again, this doesn't really matter that much right now, because files under `/etc/fish` will always be executed *after* your user config. --- There's also the issue that fish doesn't technically source /etc/fish/..., it looks in `$__fish_sysconf_dir` which is set at compile time to a location in the nix store. The only reason `/etc/fish/config.fish` is executed is [because of a shim](https://github.com/NixOS/nixpkgs/blob/7cf96f8ebb562afa7bf2f40f0d5d06cb7fe096dd/pkgs/shells/fish/default.nix#L88-L89)). But nothing knows about `/etc/fish/conf.d`, so it's not used. To fix this, maybe it's worth changing nixpkgs' fish preinit to add `/etc/fish` to `__extra_confdir`? I'm not sure what the ramifications would be for this ...
yoav-lavi commented 11 months ago

@clo4 note that if you plan on using this in a home-manager managed fish shell, you'll need to use osConfig.environment.profiles rather than config.environment.profiles (at least that's the only thing that ended up working for me)

khaneliman commented 11 months ago

@clo4 note that if you plan on using this in a home-manager managed fish shell, you'll need to use osConfig.environment.profiles rather than config.environment.profiles (at least that's the only thing that ended up working for me)

This helped me when I transitioned to using home-manager configs for my shell on my macbook. Thanks.

britter commented 8 months ago

Hey, I'm experiencing the problem with binaries installed via nix-darwin and home-manager not being present in PATH, but for me it looks like the root cause is different in my case.

Context

I'm migrating an existing homebrew setup over to a flake base nix-darwin setup with home-manager. I installed nix using the Determinate Systems installer. Right after the first nix run nix-darwin -- switch --flake . I noticed that darwin-rebuild was not in my PATH. I ignored that for the beginning and started migrating stuff from homebrew over to my new home-manager setup. After migrating my fish configuration and uninstalling the homebrew fish install, all the binaries installed via home-manager didn't work in fish anymore. In fact, there terminal app didn't work anymore because /opt/homebrew/bin/fish was removed.

Root cause

The root cause seems to be that I had configured chsh -s /opt/homebrew/bin/fish. That led to the "wrong" fish installation being started when I opened the terminal. When I start fish via /run/current-system/sw/bin/fish the path is correct. My expectation is, that nix-darwin and/or home-manager take care of configuring this for me if I set users.users.my-user.shell = pkgs.fish and programs.fish.enabled = true but it doesn't seem to be the case. Also the fish installed from nix is not present in /etc/shells and can therefore not be selected using chsh

Fix

Workaround

PATH should be correct now.

lilyball commented 8 months ago

@britter The problem is nix-darwin does not modify any user accounts unless they're listed in users.knownUsers, but that warns you not to put the admin user or other system users there because having a user listed there means nix-darwin is free to create/delete it or change whatever it wants. Also, skimming the implementation right now, I'm not sure it applies any changes to user options to an already-existing user, it looks like it just sets everything when it creates the user.

In my local setup I have my own activation script that explicitly checks for and updates my user shell because of this issue (and also this code actually sets the shell to a wrapper program that basically does /bin/wait4path /nix/store && exec $shell "$@", except it's a C program instead of a shell script wrapper to avoid issues with argv[0]).

britter commented 8 months ago

@lilyball can you point me to the documentation of users.knownUsers? I tried finding something about it but failed.

I think there are two things at play here:

  1. setting programs.fish.enable = true should add fish to etc/shells because otherwise it's unusable. On my system that didn't happen.
  2. When setting pkgs.fish as defaultShell for my user, I expect it to chsh -s to fish, so fish becomes my default shell.

This is what happens on NixOS, but it doesn't seem to work on my macOS machine. Curious to understand if I made a mistake or if I just have the wrong expectations.

lilyball commented 8 months ago

@britter

https://github.com/LnL7/nix-darwin/blob/afe83cbc2e673b1f08d32dd0f70df599678ff1e7/modules/users/default.nix#L41-L49 https://github.com/LnL7/nix-darwin/blob/afe83cbc2e673b1f08d32dd0f70df599678ff1e7/modules/users/default.nix#L137 If you look at the whole activation script here you can see that it calculates a list of "created" and "deleted" users by cross-referencing the users.knownUsers list with the users.users set. For every "created" user, if users.forceRecreate is set it first deletes the user (if it exists) and then recreates it, and if that's not set then it simply skips any user that already exists (meaning it won't apply any changes to user properties).

I feel like this whole module needs to be rewritten such that it will sync changes to known users.

For any user not listed in the known users list, the properties you can set effectively act as ways to notify nix-darwin as to what your current user config is, in case anything wants to reference that. For example, I use this to tell it where the home folder for my user is, so home-manager knows where to write its config to.

As for /etc/shells, as best I can figure, that file actually doesn't matter for most things on macOS anymore. I don't even bother updating that with my config anymore. The comment in the file as it ships with macOS says that Ftpd consults it, but I guess nothing else does.

jjant commented 7 months ago

Can we merge @clo4's fix into nix-darwin, @LnL7?

zhou13 commented 1 month ago

For people having issues with zsh, the following seems working for me:

        programs.zsh = {
          enable = true;
          loginShellInit = ". /etc/zprofile\n";
        };