opening lock file '/nix/var/nix/profiles/per-user/root/home-manager.lock': No such file or directory

emils0 commented 1 year ago

Are you following the right branch?

[X] My Nixpkgs and Home Manager versions are in sync

Is there an existing issue for this?

[X] I have searched the existing issues

Issue description

When installing home-manager on a non-NixOS Linux Distro, I am encountering an error during installation.

$ nix-shell '<home-manager>' -A install

Creating initial Home Manager generation...

/nix/store/sgbp7cxbaclil0jchnrswqhjdsp5bfv7-home-manager-generation
Starting Home Manager activation
error: opening lock file '/nix/var/nix/profiles/per-user/root/home-manager.lock': No such file or directory
Uh oh, the installation failed! Please create an issue at

    https://github.com/nix-community/home-manager/issues

if the error seems to be the fault of Home Manager.

This fails consistently, and I have attempted to install it on multiple distros inside of WSL, including Ubuntu, Debian, and Arch.

A friend of mine is also experiencing this exact issue, but he's running Fedora on bare-metal, so this does not seem to be exclusively happening on WSL.

I have also tried installing home-manager on both multi-user Nix installs, and single-user, but the result is the same.

Maintainer CC

No response

System information

- system: `"x86_64-linux"`
 - host os: `Linux 5.10.16.3-microsoft-standard-WSL2, Arch Linux, noversion, rolling`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.14.1`
 - nixpkgs: `/root/.nix-defexpr/channels/nixpkgs`

ncfavier commented 1 year ago

This is a consequence of https://github.com/nix-community/home-manager/issues/2544, which is starting to cause issues with Nix 2.14 because the default profile location changed, see https://github.com/NixOS/nix/issues/7930.

propet commented 1 year ago

I'm on a Kali linux host, and I could solve this issue by doing:

mkdir -m 0755 -p /nix/var/nix/{profiles,gcroots}/per-user/$USER

rycee commented 1 year ago

Thanks for the report! I'll try to get this resolved over the weekend.

thiagokokada commented 1 year ago

A similar issue also happens before 2.14 when you try use the store located in ~/.local/share/nix/root, that is done automatically since 2.10 if you don't have permissions to write in /nix (see: https://discourse.nixos.org/t/nix-2-10-0-released/20291). This is ideal for using Home-Manager inside systems that have / in read-only mode, like Steam Deck.

So having this bug fixed can be interesting since it would mean that we could have Home-Manager running in almost any Linux system that supports user-namespaces and have access to Nix static binaries. This also means things like https://github.com/nix-community/nix-user-chroot/ (that is unmaintained right now) wouldn't be necessary anymore (at least for the Home-Manager use case).

rycee commented 1 year ago

@thiagokokada Hmm, wouldn't setting NIX_STATE_DIR work in that case?

thiagokokada commented 1 year ago

@thiagokokada Hmm, wouldn't setting NIX_STATE_DIR work in that case?

No, for two reasons:

If you set NIX_STATE_DIR globally (like export NIX_STATE_DIR=~/.local/share/nix/var/nix, it makes nix assume that the store is once again localized in /nix/store (I think this is a bug in nix, not sure yet). Once it tries to write it fails.
If you set only for the activation script, it goes further but stops later for some other reason that I didn't had enough time to debug yet.

midirhee12 commented 1 year ago

Yeah, I also have this issue. Reproducible on Debian.

thiagokokada commented 1 year ago

With the following patch:

diff --git a/modules/lib-bash/activation-init.sh b/modules/lib-bash/activation-init.sh
index 3b0f5320..24735cc5 100644
--- a/modules/lib-bash/activation-init.sh
+++ b/modules/lib-bash/activation-init.sh
@@ -1,7 +1,16 @@
+function nixStateDir() {
+    local nixStateDir="/nix/var/nix"
+    if [[ ! -d "$nixStateDir" ]]; then
+        nixStateDir="${XDG_DATA_HOME:-$HOME/.local/share}/nix"
+    fi
+    # NIX_STATE_DIR has priority over directory auto-detection
+    echo "${NIX_STATE_DIR:-$nixStateDir}"
+}
+
 function setupVars() {
-    local nixStateDir="${NIX_STATE_DIR:-/nix/var/nix}"
-    local profilesPath="$nixStateDir/profiles/per-user/$USER"
-    local gcPath="$nixStateDir/gcroots/per-user/$USER"
+    local -r nixStateDir="$(nixStateDir)"
+    local -r profilesPath="$nixStateDir/profiles/per-user/$USER"
+    local -r gcPath="$nixStateDir/gcroots/per-user/$USER"

     declare -gr nixProfilePath="$profilesPath/profile"
     declare -gr genProfilePath="$profilesPath/home-manager"

The issue of the topic seems to be fixed, but I am not sure about the implications yet. Also this is not sufficient to make Home-Manager to work with store inside $HOME/.local/share/nix because the symlinks inside the home still assumes that everything is coming from /nix/store, so something else needs to be changed inside the code.

midirhee12 commented 1 year ago

Any update on the progress of this? All workarounds suggested are really nothing more than hacks so far. And this is keeping anyone using nix v2.14 or newer from installing home-manager. Iirc, this should also be a problem for new nixos users too, but I have not verified this. But all new nix versions will be using the new profile location, so this is a bit problematic.

midirhee12 commented 1 year ago

https://github.com/nix-community/home-manager/pull/3742

I think it is worth mentioning this pull request here.

rycee commented 1 year ago

@thiagokokada Thanks for the description! I see your point about NIX_STATE_DIR.

If anybody could try out #3742 that would be great. It seems to work OK for me in a fresh Debian VM with Nix 2.14.1 but it would be good to have extra confirmation.

rycee commented 1 year ago

@cvoges12 The stable version of Nix is still 2.13 at the moment so NixOS users should be safe.

IMHO 2.13 should stay as stable until the interaction with nix-collect-garbage is sorted out (as far as I can tell nix-collect-garbage will not expire profiles at the new profile location). Ideally also documentation updates (this pretty major change is not even mentioned in the release notes, unless I'm mistaken).

Edit: I realized that the above may sound a bit bitter but I assure it is just because I'm very sleepy 🙂 I think managing the user's profiles within the user's directory makes a lot of sense.

midirhee12 commented 1 year ago

No bitterness inferred heh. I like to be very direct as well.

I still think unstable should be prioritized. But yes, I agree that such a major change should be put in the release notes.

Anyways, I've cloned the branch that you've made. But I'm not seeing where in the contributing section of the manual on how to build it without home-manager already installed. It seems that section 5.1 assumes home-manager has already been successfully installed. I tried building the flake via:

nix build --no-link path:<path>/home-manager
"$(nix path-info path:<path>/home-manager)"/activate

Which gave an error: <shell>: /nix/store/<hash>-home-manager/activate: No such file or directory

I might be wrong in how I'm attempting this, but I do know that at least /nix/store/<hash>-home-manager does exist with a bin and share directory.

jskulski commented 1 year ago

@rycee #3742 works with test-install, and was able to install out of my git clone 👍

I had a raspberry pi running Raspbian/Debian 11 that failed finding the lock file. Google searches led me here.

jsk@fieldagent:~ $ nix --version
nix (Nix) 2.14.0
jsk@fieldagent:~ $ ls -l .nix*
-rw-r--r-- 1 jsk jsk  133 Mar  2 23:55 .nix-channels
lrwxrwxrwx 1 jsk jsk   43 Feb 28 18:20 .nix-profile -> /home/jsk/.local/state/nix/profiles/profile

I ran make test-install on a fresh clone of the repo.

On master test-install failed finding the lock file as expected.

building '/nix/store/25x60nr8kn99jrhajn81gdnnl5kih69x-activation-script.drv'...
building '/nix/store/0rswr5gllp8g4niyg4sbplp6kwbjfsfs-home-manager-generation.drv'...
/nix/store/25d890qaafdi6npj75kdj4c1107xw35l-home-manager-generation
/nix/store/fz80arp0cldc6lprk1z11g3d62vpmj78-bash-5.2-p15/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
Starting Home Manager activation
error: opening lock file '/nix/var/nix/profiles/per-user/jsk/home-manager.lock': No such file or directory
Uh oh, the installation failed! Please create an issue at

    https://github.com/nix-community/home-manager/issues

if the error seems to be the fault of Home Manager.
make: *** [Makefile:17: test-install] Error 1

jsk@fieldagent:~/Code/github/nix-community/home-manager $ git branch
* master
  migrate-profiles-and-gcroot

And make test-install on the migrate-profiles-and-gcroot branch:

installing 'home-manager-path'
building '/nix/store/q572n1dx7v725vr0c7im1194cqnrw0ya-user-environment.drv'...
Activating linkGeneration
Creating profile generation 1
Creating home file links in /tmp/tmp.oQeM5040Vn
bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
Activating onFilesChange
Activating reloadSystemd
bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

There are 84 unread and relevant news items.
Read them by running the command "home-manager news".

All done! The home-manager tool should now be installed and you can edit

    /tmp/tmp.oQeM5040Vn/.config/nixpkgs/home.nix

to configure Home Manager. Run 'man home-configuration.nix' to
see all available options.

jsk@fieldagent:~/Code/github/nix-community/home-manager $ git branch
  master
* migrate-profiles-and-gcroot

~I haven't installed it fully on my system yet because well I haven't figured it out yet, but let me know if you need any more info~

Update: Confirmed install. Learned what "" means and ran:

nix-shell '/home/jsk/Code/github/nix-community/home-manager/' -A install`

which followed the same behavior (fails on main, OK on branch).

~Now, I have a working home-manager~ 🎉

--

UPDATE 2:

I maybe running into an issue with home-manager switch.

jsk@fieldagent:~ $ home-manager switch -v
/nix/store/h2b36gwxhryrzb1hs81shhj6grm0fk14-home-manager-generation
Starting Home Manager activation
Sanity checking Nix
error: opening lock file '/nix/var/nix/profiles/per-user/jsk/home-manager.lock': No such file or directory

I may have something backwards. I have programs.home-manager.path set, but I am not seeing the changes I made in the repository being reflected, so haven't been able to debug. I did try was running all of them:

jsk@fieldagent:/nix/store $ _i() {  echo $*; }
jsk@fieldagent:/nix/store $ for i in `find *home-manager* -type f -name 'activation-init.sh'`; do echo "== $i =="; echo; source $i; echo; done

== 71qi4gd7snhvr2yfw79qnh9502aa3aa0-home-manager/home-manager/modules/lib-bash/activation-init.sh ==

Starting Home Manager activation
-bash: declare: nixProfilePath: readonly variable
-bash: declare: genProfilePath: readonly variable
-bash: declare: newGenPath: readonly variable
-bash: declare: newGenGcPath: readonly variable
error: opening lock file '/nix/var/nix/profiles/per-user/jsk/home-manager.lock': No such file or directory
-bash: declare: newGenNum: readonly variable

== agkiji6c3yq3mfh1grmfx6cizvi6dl44-home-manager/home-manager/modules/lib-bash/activation-init.sh ==

Starting Home Manager activation
-bash: declare: nixProfilePath: readonly variable
-bash: declare: genProfilePath: readonly variable
-bash: declare: newGenPath: readonly variable
-bash: declare: newGenGcPath: readonly variable
error: opening lock file '/nix/var/nix/profiles/per-user/jsk/home-manager.lock': No such file or directory
-bash: declare: newGenNum: readonly variable

So it looks like there's another reference somewhere, or I have it backwards.

rycee commented 1 year ago

@jskulski Hmm, it may be that you got an older commit. I think the readonly variable issues should be resolved in the latest version.

I merged the PR to master so now it should be possible to test just by updating HM in the regular way.

jskulski commented 1 year ago

Thanks, installed and everything working as expected for me.

rycee commented 1 year ago

@jskulski Great to hear! Thanks for trying out and reporting on the result.

I'll close this issue for now, if anybody encounters the same issue after updating please write a comment.

Lindenk commented 1 year ago

I seem to be having the same issue, also on Arch Linux. My .nix-profile points to /home/lindenk/.local/state/nix/profiles/profile, and nix-env works using it, however home-manager specifically tries to lock /nix/var/nix/profiles/per-user/lindenk/home-manager.lock which does not exist. Creating the directories manually causes Permission denied. This is also an issue using both the latest unstable (23.05-pre) and github:nix-community/home-manager/release-22.11

For reference I'm trying to create dotfiles with a nix flake, with pretty much the same template as this. Running home-manager switch --flake '.#lindenk' causes the issue

EDIT: The guide uses release-22.11 which has this bug. switching to unstable worked

nixos-discourse commented 1 year ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/error-opening-lock-file-nix-var-nix-profiles-per-user-myuser-home-manager-lock-permission-denied/27930/2

ulysses4ever commented 1 year ago

For reference I'm trying to create dotfiles with a nix flake, with pretty much the same template as this. Running home-manager switch --flake '.#lindenk' causes the issue

I'm trying the same and running into this issue on Ubuntu 20.04. I (re)installed nix yesterday using the sh-based command from the website. And I have the same permission denied failure. The profiles directory does exist.

> nix run home-manager/master -- --flake ~/dotfiles/hm-only switch
error: opening lock file '/nix/var/nix/profiles/per-user/artem/home-manager.lock': Permission denied

> ll /nix/var/nix/profiles/per-user/artem
lrwxrwxrwx 15 root  6 May 16:39 channels -> channels-4-link
lrwxrwxrwx 60 root 11 Dec  2020 channels-2-link -> /nix/store/qni5gy8rjjyw3vpqh0ij10mq6h52x206-user-environment
lrwxrwxrwx 60 root  8 Oct  2021 channels-3-link -> /nix/store/95wnq250bhn6jk73ivd67s6dh2l3v85g-user-environment
lrwxrwxrwx 60 root  6 May 16:39 channels-4-link -> /nix/store/7ibvmbxqls2310lkflls4ibxrpwk84ms-user-environment
lrwxrwxrwx 14 root  6 May 16:41 profile -> profile-2-link
lrwxrwxrwx 60 root 27 Apr  2020 profile-1-link -> /nix/store/04z1n36fhpdr0fayp2s2v3fp3njvsr79-user-environment
lrwxrwxrwx 60 root  6 May 16:41 profile-2-link -> /nix/store/j8dis32qf9hmfbgg1gb58iz8ara328q6-user-environment

Could someone help me figuring what to do about it? Should I provide more info? My config (including the hm-only config mentioned above) is on the branch here: https://github.com/ulysses4ever/dotfiles/tree/prl-julia. Here's nix-info:

> nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.4.0-122-generic, Ubuntu, 20.04.4 LTS (Focal Fossa), nobuild`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.13.3`
 - channels(artem): `"nixpkgs-23.05pre481435.caf436a52b2"`
 - channels(root): `"nixpkgs"`
 - nixpkgs: `/home/artem/.nix-defexpr/channels/nixpkgs`

nixos-discourse commented 1 year ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/error-opening-lock-file-nix-var-nix-profiles-per-user-myuser-home-manager-lock-permission-denied/27930/4

nuttycom commented 1 year ago

I just ran into this again on a fresh nix install using version 2.16.1 on Ubuntu 23.04 (lunar).

ethanabrooks commented 1 year ago

I am also having this issue. Same setup as @nuttycom. Please reopen.

Current workaround for multi-user install is to uninstall nix and reinstall version 2.13.0 using:

 curl -L https://releases.nixos.org/nix/nix-2.13.0/install | sh -s -- --daemon

emagdel commented 1 year ago

I'm on a Kali linux host, and I could solve this issue by doing:
mkdir -m 0755 -p /nix/var/nix/{profiles,gcroots}/per-user/$USER

Similarly, the following worked for me: sudo chown -R $USER:nixbld /nix/var/nix/profiles/per-user/$USER

coolcorexix commented 1 year ago

I'm on a Kali linux host, and I could solve this issue by doing:
mkdir -m 0755 -p /nix/var/nix/{profiles,gcroots}/per-user/$USER
Similarly, the following worked for me: sudo chown -R $USER:nixbld /nix/var/nix/profiles/per-user/$USER

this work like a charm, thank you

jjant commented 9 months ago

Just ran into this problem with a fresh Mac.

System information:

OS: macOS Sonoma 14.3
Nix: nix (Nix) 2.20.1
HomeManager: 24.05-pre

I had to do both:

mkdir -m 0755 -p /nix/var/nix/{profiles,gcroots}/per-user/$USER
sudo chown -R $USER:nixbld /nix/var/nix/{profiles,gcroots}/per-user/$USER

to get HM switch to work.

thiagokokada commented 9 months ago

Just ran into this problem with a fresh Mac.

System information:
OS: macOS Sonoma 14.3
Nix: nix (Nix) 2.20.1
HomeManager: 24.05-pre
I had to do both:
mkdir -m 0755 -p /nix/var/nix/{profiles,gcroots}/per-user/$USER
sudo chown -R $USER:nixbld /nix/var/nix/{profiles,gcroots}/per-user/$USER
to get HM switch to work.

Looks to me that Home-Manager itself is failing to create the profile directory in the user home if it doesn't exist.

CC @rycee.

drupol commented 6 months ago

On Ubuntu:

sudo mkdir -m 0755 -p /nix/var/nix/{profiles,gcroots}/per-user/$USER
sudo chown -R $USER:nixbld /nix/var/nix/{profiles,gcroots}/per-user/$USER

Worked perfectly. I wonder why we have to do that... UX wise it's very repellent.

nix-community / home-manager