bug: home-manager does not work with store outside `/nix/store`

[thiagokokada]

Are you following the right branch?

• [X] My Nixpkgs and Home Manager versions are in sync

Is there an existing issue for this?

• [X] I have searched the existing issues

Issue description

Since Nix 2.10, it is possible to use Nix on Linux systems without having root permissions to write in / by pointing Nix to $XDG_USER_DATA/nix/root (generally ~/.local/share/nix/root) instead. This is an interesting use case for users that don't have write access to /, for example computer schools or Steam Deck.

3742 is a good start point for this issue, fixing the profile and GC roots to allow them to work from $XDG_USER_DATA/home-manager/ instead, but additional work is needed so that Home-Manager creates everything in the correct place, e.g. the symlinks needs to come from $XDG_USER_DATA/nix/root instead of assuming they're coming from /nix/store.

A fresh install of Fedora Workstation 37 with Nix 2.14.1 static binaries[1][2][3] and my personal configuration repo results in the following error as commit https://github.com/nix-community/home-manager/commit/36999b8d19eb6eebb41983ef017d7e0095316af2:

$ nix run .#homeActivations/home-linux-minimal --extra-experimental-features 'nix-command flakes'
warning: Git tree '/home/thiagoko/Projects/nix-configs' is dirty
Starting Home Manager activation
Activating checkFilesChanged
Activating checkLinkTargets
Activating writeBoundary
Activating installPackages
installing 'home-manager-path'
don't know how to build these paths:
  /nix/store/bms0bf7dhxdh79kyhavxxn1zqqag5ziw-home-manager-path
error: path '/nix/store/bms0bf7dhxdh79kyhavxxn1zqqag5ziw-home-manager-path' does not exist and cannot be created

Oops, Nix failed to install your new Home Manager profile!

Perhaps there is a conflict with a package that was installed using
"nix-env -i"? Try running

    nix-env -q

and if there is a conflicting package you can remove it with

    nix-env -e {package name}

Then try activating your Home Manager configuration again.

1. Here it is the Nix static binary to make reproducibility of this bug easier: nix-static-x86_64-unknown-linux-musl-2.14.1.zip. It is a tar.xz inside a zip file because of GitHub limitation.
2. You can just add the bin directory from the Nix static to your PATH, and nix should work as expected (expect that there is no NIX_PATH set, but Flakes works as it should).
3. For those who want to compile it themselves, just run nix build .#legacyPackages.x86_64-linux.pkgsStatic.nixUnstable inside a fairly recent nixpkgs commit. You may need to disable the installCheckPhase though (see: https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/package-management/nix/common.nix#L200).

Maintainer CC

@berbiche

System information

[thiagoko@localhost-live nix-configs]$ nix shell nixpkgs#nix-info --extra-experimental-features 'nix-command flakes' -c nix-info -m
 - system: `"x86_64-linux"`
 - host os: `Linux 6.0.7-301.fc37.x86_64, Fedora Linux, 37 (Workstation Edition), nobuild`
 - multi-user?: `no`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.14.1`
find: '/nix/var/nix/profiles/per-user': No such file or directory

[rycee]

Thanks for the report! It would be very nice if we could support this feature.

About the specific error you encounter, it seems to happen when doing nix-env -i ${cfg.path}, where ${cfg.path} is a derivation of the form pkgs.buildEnv { … }. So for some reason, it seems like ${cfg.path} expands to a path in /nix/store, not the one in the home directory.

I'm entirely unfamiliar with this "store in $HOME" feature so I have no idea how it is actually implemented. I guess it relies on some kind of chrooting and therefore the /nix/store path is valid within the chroot? And the activation script somehow runs in a context where the /nix/store path is not pointing to the directory inside $HOME?

[thiagokokada]

I'm entirely unfamiliar with this "store in $HOME" feature so I have no idea how it is actually implemented. I guess it relies on some kind of chrooting and therefore the /nix/store path is valid within the chroot? And the activation script somehow runs in a context where the /nix/store path is not pointing to the directory inside $HOME?

This looks like correct. I am also not entirely familiar with how the feature works, but looking at the commit that introduced it this looks how it is implemented: https://github.com/NixOS/nix/commit/2a9fddc0b16d9b4771d11fc10d8b2a9cba55ff64#diff-e89a2224d1d4c39bf5a2d448b7b2fac1e3e75c7e08e414ee448846266c8bfeabR1323-R1334

[thiagokokada]

About the specific error you encounter, it seems to happen when doing nix-env -i ${cfg.path}, where ${cfg.path} is a derivation of the form pkgs.buildEnv { … }. So for some reason, it seems like ${cfg.path} expands to a path in /nix/store, not the one in the home directory.

So changing this line like you mentioned and hardcoding the path with this patch:

diff --git a/modules/home-environment.nix b/modules/home-environment.nix
index aca3723e..05206cda 100644
--- a/modules/home-environment.nix
+++ b/modules/home-environment.nix
@@ -620,7 +620,7 @@ in
             REMOVE_CMD_SYNTAX='nix-env -e {package name}'
           fi

-          if ! $INSTALL_CMD_ACTUAL ${cfg.path} ; then
+          if ! $INSTALL_CMD_ACTUAL ~/.local/share/nix/root/${cfg.path} ; then
             echo
             _iError $'Oops, Nix failed to install your new Home Manager profile!\n\nPerhaps there is a conflict with a package that was installed using\n"%s"? Try running\n\n    %s\n\nand if there is a conflicting package you can remove it with\n\n    %s\n\nThen try activating your Home Manager configuration again.' "$INSTALL_CMD" "$LIST_CMD" "$REMOVE_CMD_SYNTAX"

Results in the following error:

$ nix run .#homeActivations/home-linux-minimal --extra-experimental-features 'nix-command flakes' --override-input home ~/Projects/home-manager/
warning: Git tree '/home/thiagoko/Projects/nix-configs' is dirty
warning: Git tree '/home/thiagoko/Projects/home-manager' is dirty
warning: not writing modified lock file of flake 'git+file:///home/thiagoko/Projects/nix-configs':
• Updated input 'home':
    'github:nix-community/home-manager/36999b8d19eb6eebb41983ef017d7e0095316af2' (2023-03-08)
  → 'git+file:///home/thiagoko/Projects/home-manager' (2023-03-08)
Starting Home Manager activation
Activating checkFilesChanged
Activating checkLinkTargets
Activating writeBoundary
Activating installPackages
error: path '/home/thiagoko/.local/share/nix/root/nix/store/bms0bf7dhxdh79kyhavxxn1zqqag5ziw-home-manager-path' is not in the Nix store

Oops, Nix failed to install your new Home Manager profile!

Perhaps there is a conflict with a package that was installed using
"nix-env -i"? Try running

    nix-env -q

and if there is a conflicting package you can remove it with

    nix-env -e {package name}

Then try activating your Home Manager configuration again.

[thiagokokada]

Huh... I see. I think the issue is that Nix with store from $XDG_DATA_DIR/nix/root does not work well with the old CLI. Forcing the new CLI seems to get me further:

diff --git a/modules/home-environment.nix b/modules/home-environment.nix
index aca3723e..c53903a4 100644
--- a/modules/home-environment.nix
+++ b/modules/home-environment.nix
@@ -608,17 +608,10 @@ in
             $DRY_RUN_CMD $oldNix profile install $1
           }

-          if [[ -e $HOME/.nix-profile/manifest.json ]] ; then
-            INSTALL_CMD="nix profile install"
-            INSTALL_CMD_ACTUAL="nixReplaceProfile"
-            LIST_CMD="nix profile list"
-            REMOVE_CMD_SYNTAX='nix profile remove {number | store path}'
-          else
-            INSTALL_CMD="nix-env -i"
-            INSTALL_CMD_ACTUAL="$DRY_RUN_CMD nix-env -i"
-            LIST_CMD="nix-env -q"
-            REMOVE_CMD_SYNTAX='nix-env -e {package name}'
-          fi
+          INSTALL_CMD="nix profile install"
+          INSTALL_CMD_ACTUAL="nixReplaceProfile"
+          LIST_CMD="nix profile list"
+          REMOVE_CMD_SYNTAX='nix profile remove {number | store path}'

           if ! $INSTALL_CMD_ACTUAL ${cfg.path} ; then
             echo

The result:

$ nix run .#homeActivations/home-linux-minimal --extra-experimental-features 'nix-command flakes' --override-input home ~/Projects/home-manager/
warning: Git tree '/home/thiagoko/Projects/nix-configs' is dirty
warning: Git tree '/home/thiagoko/Projects/home-manager' is dirty
warning: not writing modified lock file of flake 'git+file:///home/thiagoko/Projects/nix-configs':
• Updated input 'home':
    'github:nix-community/home-manager/36999b8d19eb6eebb41983ef017d7e0095316af2' (2023-03-08)
  → 'git+file:///home/thiagoko/Projects/home-manager' (2023-03-08)
Starting Home Manager activation
Activating checkFilesChanged
Activating checkLinkTargets
Activating writeBoundary
Activating installPackages
nix profile remove
warning: Use 'nix profile list' to see the current profile.
error: path '/nix/store/bms0bf7dhxdh79kyhavxxn1zqqag5ziw-home-manager-path' is required, but there is no substituter that can build it

Oops, Nix failed to install your new Home Manager profile!

Perhaps there is a conflict with a package that was installed using
"nix profile install"? Try running

    nix profile list

and if there is a conflicting package you can remove it with

    nix profile remove {number | store path}

Then try activating your Home Manager configuration again.

This is starting to get interesting, but still not working.

[thiagokokada]

So I think I am starting to understand more how this feature works. If we start a nix shell, we can see that /nix is "mounted" inside it:

$ nix shell nixpkgs#bash -c bash -l
bash: shopt: progcomp: invalid shell option name
$ ls -lah /nix
total 0
drwxr-xr-x.  3 thiagoko thiagoko   60 Mar  9 00:25 .
drwxr-xr-x. 20 thiagoko thiagoko  480 Mar  9 00:25 ..
drwxr-xr-x.  1 thiagoko thiagoko 534K Mar  9 00:23 store

Sadly trying to activate my Home-Manager inside this environment does not work:

$ nix run .#homeActivations/home-linux-minimal --extra-experimental-features 'nix-command flakes'
error: setting up a private mount namespace: Operation not permitted
$ nix shell nixpkgs#bash
error: setting up a private mount namespace: Operation not permitted

Probably because this is using user-namespace and AFAIK user-namespaces does not allow you to mount a user-namespace inside itself.

[hurricanehrndz]

@thiagokokada

I got home-manager to work, this is what I ended up doing:

1. Setup nix

   mkdir -p ~/.config/nix
   echo "experimental-features = nix-command flakes" >> ~/.config/nix/nix.conf
   mkdir -p ~/.local/bin
   curl -o ~/.local/bin/nix -L https://hydra.nixos.org/job/nix/master/buildStatic.x86_64-linux/latest/download/1
   chmod +x ~/.local/bin/nix
   export PATH=~/.local/bin:$PATH

2. Build home activation package, i.e.

   cd ~/homeconfig
   nix build .#homeConfigurations.x86_64-linux.traveller.activationPackage

3. Start nix shell with nix

   nix shell nixpkgs#nix --command bash --noprofile --norc -l
   cd ~/homeconfig
   ./result/bin/home-manager-generation

   Then you should be able run you configured shell i.e

exit
nix run nixpkgs#zsh

[hurricanehrndz]

I also ended up adding the following script to the host zshrc so that I am launched directly into nix zsh shell

 script to start zsh in chroot store
NIX=$HOME/.local/tmp/nix
export TERM=xterm-256color
export PATH=$HOME/.nix-profile/bin:$PATH
# fix dobule chars
# see https://github.com/ohmyzsh/ohmyzsh/issues/7426
export LANG=C.UTF-8
$NIX shell nixpkgs#nix nixpkgs#zsh --command zsh

[stale[bot]]

Thank you for your contribution! I marked this issue as stale due to inactivity. Please be considerate of people watching this issue and receiving notifications before commenting 'I have this issue too'. We welcome additional information that will help resolve this issue. Please read the relevant sections below before commenting.

If you are the original author of the issue

* If this is resolved, please consider closing it so that the maintainers know not to focus on this. * If this might still be an issue, but you are not interested in promoting its resolution, please consider closing it while encouraging others to take over and reopen an issue if they care enough. * If you know how to solve the issue, please consider submitting a Pull Request that addresses this issue.

If you are not the original author of the issue

* If you are also experiencing this issue, please add details of your situation to help with the debugging process. * If you know how to solve the issue, please consider submitting a Pull Request that addresses this issue.

Memorandum on closing issues

Don't be afraid to manually close an issue, even if it holds valuable information. Closed issues stay in the system for people to search, read, cross-reference, or even reopen – nothing is lost! Closing obsolete issues is an important way to help maintainers focus their time and effort.

[obff]

Hey there,

I roughly followed the steps described by @hurricanehrndz. Now I am able to connect to some remote host via ssh -t host.example.com '.local/bin/nix run nixpkgs/nixos-23.11#bashInteractive'. After doing so I end up in my home-manager controlled bash environment.

If I try to run nix develop for some flake (e.g. nix develop nixpkgs#hello), I get the following error message.

error: setting up a private mount namespace: Operation not permitted

If I first connect via ssh to the remote host without entering the home-manger controlled bash environment, I am able to enter the development shell via .local/bin/nix develop nixpkgs#hello. I still get some error.

error (ignored): error: package 'nixpkgs#bashInteractive' does not provide a 'bin/bash'

But finally I end up in the anticipated development shell in my home-manager controlled bash environment.

I already verified that kernel.unprivileged_userns_clone=1.

Anyone else facing this problem?

[patrick-ovh]

I roughly followed the steps described by @hurricanehrndz. Now I am able to connect to some remote host via ssh -t host.example.com '.local/bin/nix run nixpkgs/nixos-23.11#bashInteractive'. After doing so I end up in my home-manager controlled bash environment.

If I try to run nix develop for some flake (e.g. nix develop nixpkgs#hello), I get the following error message.

error: setting up a private mount namespace: Operation not permitted

This is because when you ran .local/bin/nix run you already switched to a new namespace and you can't do it again. https://github.com/NixOS/nix/blob/4072a8fea073c33b729ba77e97b0c8cf9b40a628/src/nix/run.cc#L225

If I first connect via ssh to the remote host without entering the home-manger controlled bash environment, I am able to enter the development shell via .local/bin/nix develop nixpkgs#hello. I still get some error.

error (ignored): error: package 'nixpkgs#bashInteractive' does not provide a 'bin/bash'

But finally I end up in the anticipated development shell in my home-manager controlled bash environment.

I already verified that kernel.unprivileged_userns_clone=1.

Anyone else facing this problem?

Your can try the workarounds listed in this comment: https://github.com/NixOS/nix/issues/6091#issuecomment-1037219994

I personally use third one a bit like you did:

nix shell nixpkgs/23.11#bashInteractive --offline --command bash --login
source <(nix print-dev-env nixpkgs#hello)

[obff]

Your can try the workarounds listed in this comment: NixOS/nix#6091 (comment)

Thank you very much for the pointer. Workaround 1. and 3. solve my issues :)

[txomon]

@hurricanehrndz nix newbie here, in regards to the homeconfig folder, what does it contain? I'm trying to get home-manager to work in a rootless / nix-less system and found your proposed solution, however I am failing to find what the "homeconfig" folder you speak about contains.

Would you mind sharing how starting from an empty user would set up the environment? Just in case, the following is what I did to get an unpriviledged user:

useradd -m nixtest
su -Pl nixtest

[stale[bot]]

Thank you for your contribution! I marked this issue as stale due to inactivity. Please be considerate of people watching this issue and receiving notifications before commenting 'I have this issue too'. We welcome additional information that will help resolve this issue. Please read the relevant sections below before commenting.

If you are the original author of the issue

* If this is resolved, please consider closing it so that the maintainers know not to focus on this. * If this might still be an issue, but you are not interested in promoting its resolution, please consider closing it while encouraging others to take over and reopen an issue if they care enough. * If you know how to solve the issue, please consider submitting a Pull Request that addresses this issue.

If you are not the original author of the issue

* If you are also experiencing this issue, please add details of your situation to help with the debugging process. * If you know how to solve the issue, please consider submitting a Pull Request that addresses this issue.

Memorandum on closing issues

Don't be afraid to manually close an issue, even if it holds valuable information. Closed issues stay in the system for people to search, read, cross-reference, or even reopen – nothing is lost! Closing obsolete issues is an important way to help maintainers focus their time and effort.