Open sarim opened 1 year ago
@OneBlue - another one related to your /run/usr/
Thank you for reporting this @sarim.
Interestingly, I can't reproduce the issue. Can you share the output of mount
before and after opening WSL with the gittu user ?
WSL does mount an overlayfs on /run/user/X when the session is created, but happens regardless of whether GUI apps are enabled or not so I wonder if there's something else happening here.
[wsl2]
kernelCommandLine=cgroup_no_v1=all
memory=16GB
swap=0
guiApplications=true
debugConsole=false
#vmIdleTimeout=-1
networkingMode=bridged
vmSwitch=WSLBridged
dhcp=false
macAddress=0E:00:00:00:00:00
ipv6=true
[user]
default=gittu
[boot]
systemd=true
[network]
hostname = GITTUW11WSL
generateResolvConf=false
cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0
After wsl --shutdown, log into wsl via root user with wsl -u root -e /bin/bash
. Then mount and save output. Then I open a new tab in terminal, it logs into gittu user. Run mount and save output. The two txt are attached. Here's the diff.
41a42,45
> none on /mnt/wslg/run/user/1000/rw type tmpfs (rw,relatime)
> none on /run/user/1000/rw type tmpfs (rw,relatime)
> none on /mnt/wslg/run/user/1000 type overlay (rw,relatime,lowerdir=/mnt/wslg/runtime-dir,upperdir=/mnt/wslg/run/user/1000/rw/upper,workdir=/mnt/wslg/run/user/1000/rw/work)
> none on /run/user/1000 type overlay (rw,relatime,lowerdir=/mnt/wslg/runtime-dir,upperdir=/mnt/wslg/run/user/1000/rw/upper,workdir=/mnt/wslg/run/user/1000/rw/work)
mount-after.txt mount-before.txt
Btw if I run sudo systemctl restart user@1000
, the /run/user/1000/
directory gets restored. Both wslg sockets and systemd sockets and other files are now present here. Thats what I've been doing, after starting wsl, I run the command once. Let me know if you need any more info @OneBlue
Also this might also be relevant.
root@GITTUW11WSL:~# ls /run/user/1000/rw
ls: cannot access '/run/user/1000/rw': No such file or directory
root@GITTUW11WSL:~# ls /mnt/wslg/run/user/1000/rw
ls: cannot access '/mnt/wslg/run/user/1000/rw': No such file or directory
root@GITTUW11WSL:~# ls /mnt/wslg/run/user/1000
dbus-1 pulse wayland-0 wayland-0.lock
Edit:
WSL does mount an overlayfs on /run/user/X when the session is created, but happens regardless of whether GUI apps are enabled or not so I wonder if there's something else happening here.
Umm I don't understand, when guiApplications=false, the "system"/"wslg" distro doesn't get launched. This "/mnt/wslg" directory is shared with that system distro right? So the behavior is definitely changing from how much I understand.
Below outputs are when guiApplications=false. Notice the output of mount is absent of any /mnt/wslg related entry. Though I don't understand how /mnt/wslg directory is created now as there's no such entry in mount output.
↪ ~ ➤ sudo tree /mnt/wslg/
/mnt/wslg/
└── run
└── user
└── 1000
3 directories, 0 files
↪ ~ ➤ ls /run/user/1000/
bus dbus-1 gnupg pipewire-0 pipewire-0.lock pk-debconf-socket podman systemd
↪ ~ ➤ mount
none on /mnt/wsl type tmpfs (rw,relatime)
none on /usr/lib/wsl/drivers type 9p (ro,nosuid,nodev,noatime,dirsync,aname=drivers;fmask=222;dmask=222,mmap,access=client,msize=65536,trans=fd,rfd=7,wfd=7)
/dev/sdb on / type ext4 (rw,relatime,discard,errors=remount-ro,data=ordered)
none on /usr/lib/wsl/lib type overlay (rw,nosuid,nodev,noatime,lowerdir=/gpu_lib_packaged:/gpu_lib_inbox,upperdir=/gpu_lib/rw/upper,workdir=/gpu_lib/rw/work)
rootfs on /init type rootfs (ro,size=8186832k,nr_inodes=2046708)
none on /dev type devtmpfs (rw,nosuid,relatime,size=8186860k,nr_inodes=2046715,mode=755)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,noatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,noatime)
devpts on /dev/pts type devpts (rw,nosuid,noexec,noatime,gid=5,mode=620,ptmxmode=000)
none on /run type tmpfs (rw,nosuid,nodev,mode=755)
none on /run/lock type tmpfs (rw,nosuid,nodev,noexec,noatime)
none on /run/shm type tmpfs (rw,nosuid,nodev,noatime)
none on /dev/shm type tmpfs (rw,nosuid,nodev,noatime)
none on /run/user type tmpfs (rw,nosuid,nodev,noexec,noatime,mode=755)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
tmpfs on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,relatime,mode=755)
cgroup2 on /sys/fs/cgroup/unified type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate)
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate)
drvfs on /mnt/c type 9p (rw,noatime,dirsync,aname=drvfs;path=C:\;uid=1000;gid=1000;symlinkroot=/mnt/,mmap,access=client,msize=262144,trans=virtio)
drvfs on /mnt/d type 9p (rw,noatime,dirsync,aname=drvfs;path=D:\;uid=1000;gid=1000;symlinkroot=/mnt/,mmap,access=client,msize=262144,trans=virtio)
drvfs on /mnt/e type 9p (rw,noatime,dirsync,aname=drvfs;path=E:\;uid=1000;gid=1000;symlinkroot=/mnt/,mmap,access=client,msize=262144,trans=virtio)
drvfs on /mnt/h type 9p (rw,noatime,dirsync,aname=drvfs;path=H:\;uid=1000;gid=1000;symlinkroot=/mnt/,mmap,access=client,msize=262144,trans=virtio)
drvfs on /mnt/i type 9p (rw,noatime,dirsync,aname=drvfs;path=I:\;uid=1000;gid=1000;symlinkroot=/mnt/,mmap,access=client,msize=262144,trans=virtio)
/dev/sdb on /run/user type ext4 (rw,relatime,discard,errors=remount-ro,data=ordered)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
tmpfs on /run/qemu type tmpfs (rw,nosuid,nodev,relatime,mode=755)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime)
tracefs on /sys/kernel/tracing type tracefs (rw,nosuid,nodev,noexec,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime)
tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,size=1638024k,nr_inodes=409506,mode=700,uid=1000,gid=1000)
For those that need guiApplications=true
, one of these might fix things for you until someone gets this fixed in WSL:
sudo systemctl restart user@1000
or
mkdir ~/run_user_1000
mv /run/user/1000/* ~/run_user_1000/
sudo umount /run/user/1000
mv ~/run_user_1000/* /run/user/1000/
rm -rf ~/run_user_1000/
I just wrote and add this to my bashrc to restart user session. the command is allowed in sudoers file.
function check_and_restart_session {
# Check if "/run/user/1000/bus" exists
if [ -e "/run/user/1000/bus" ]; then
return 0
fi
# Try to avoid race condition
sleep 0.$(( ( RANDOM % 300 ) + 50 ))
# Check if "/tmp/gittuRestartSession" exists
if [ -e "/tmp/gittuRestartSession" ]; then
echo "/tmp/gittuRestartSession exists"
return 0
fi
# If neither condition is true, restart the session
touch /tmp/gittuRestartSession
sudo /usr/bin/systemctl restart user@1000.service
echo "Restart User Session"
}
This just started happening to me today, where I actually have a working system for a bit but then it breaks. In journalctl -b0
I can see:
Sep 19 12:14:55 JABAILE-DESK02 systemd[1]: dmesg.service: Deactivated successfully.
Sep 19 12:14:55 JABAILE-DESK02 sudo[736]: jabaile : TTY=pts/0 ; PWD=/home/jabaile ; USER=root ; COMMAND=/usr/bin/ls -lh /run/user
Sep 19 12:14:55 JABAILE-DESK02 sudo[736]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1000)
Sep 19 12:14:55 JABAILE-DESK02 sudo[736]: pam_unix(sudo:session): session closed for user root
Sep 19 12:15:05 JABAILE-DESK02 sudo[747]: jabaile : TTY=pts/0 ; PWD=/home/jabaile ; USER=root ; COMMAND=/usr/bin/ls -lh /run/user
Sep 19 12:15:05 JABAILE-DESK02 sudo[747]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1000)
Sep 19 12:15:05 JABAILE-DESK02 sudo[747]: pam_unix(sudo:session): session closed for user root
Sep 19 12:15:09 JABAILE-DESK02 systemd[1]: systemd-timedated.service: Deactivated successfully.
Sep 19 12:15:11 JABAILE-DESK02 sudo[773]: jabaile : TTY=pts/0 ; PWD=/home/jabaile ; USER=root ; COMMAND=/usr/bin/ls -lh /run/user
Sep 19 12:15:11 JABAILE-DESK02 sudo[773]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1000)
Sep 19 12:15:11 JABAILE-DESK02 sudo[773]: pam_unix(sudo:session): session closed for user root
Sep 19 12:15:22 JABAILE-DESK02 kernel: hv_balloon: Max. dynamic memory size: 32652 MB
Sep 19 12:16:37 JABAILE-DESK02 systemd-networkd-wait-online[187]: Timeout occurred while waiting for network connectivity.
Sep 19 12:16:37 JABAILE-DESK02 systemd[1]: systemd-networkd-wait-online.service: Main process exited, code=exited, status=1/FAILURE
Sep 19 12:16:37 JABAILE-DESK02 systemd[1]: systemd-networkd-wait-online.service: Failed with result 'exit-code'.
Sep 19 12:16:37 JABAILE-DESK02 systemd[1]: Failed to start Wait for Network to be Configured.
I'm running sudo ls -lh /run/user
and the first ones show 1000
. But then system finishes starting up and then everything that uses this dir breaks.
Running sudo systemctl restart user@1000
works, though this means I have to wait for the failure to occur and then run that command and ensure I have restarted all of my terminals, as I have various programs like fnm
which use that dir.
I thought this might be due to 2.0.0.0, but I don't have it yet:
WSL version: 1.2.2.0
Kernel version: 5.15.90.1
Perhaps this is a different issue than this thread; apologies if it is, but this is perfectly reproducible for me so I'm totally happy to try things out.
Not sure if it is the same issue I see. Right after startup is finished WSL seems to stop the User Manager. This is also visible in syslog:
Sep 21 07:32:51 w00wmi systemd[1]: Startup finished in 2min 1.606s.
Sep 21 07:33:00 w00wmi systemd[1]: Stopping User Manager for UID 1000...
Sep 21 07:33:00 w00wmi systemd[1828]: Stopped target Main User Target.
Sep 21 07:33:00 w00wmi systemd[1828]: Stopping D-Bus User Message Bus...
Sep 21 07:33:00 w00wmi systemd[1828]: Stopping PipeWire Media Session Manager...
Sep 21 07:33:00 w00wmi systemd[1828]: Stopped D-Bus User Message Bus.
Sep 21 07:33:00 w00wmi systemd[1828]: Stopped PipeWire Media Session Manager.
Sep 21 07:33:00 w00wmi systemd[1828]: Stopping PipeWire Multimedia Service...
Sep 21 07:33:00 w00wmi systemd[1828]: Stopped PipeWire Multimedia Service.
Sep 21 07:33:00 w00wmi systemd[1828]: Removed slice User Core Session Slice.
Sep 21 07:33:00 w00wmi systemd[1828]: Stopped target Basic System.
Sep 21 07:33:00 w00wmi systemd[1828]: Stopped target Paths.
Sep 21 07:33:00 w00wmi systemd[1828]: Stopped target Sockets.
Sep 21 07:33:00 w00wmi systemd[1828]: Stopped target Timers.
Sep 21 07:33:00 w00wmi systemd[1828]: Closed D-Bus User Message Bus Socket.
Sep 21 07:33:00 w00wmi systemd[1828]: Closed GnuPG network certificate management daemon.
Sep 21 07:33:00 w00wmi systemd[1828]: Closed GnuPG cryptographic agent and passphrase cache (access for web browsers).
Sep 21 07:33:00 w00wmi systemd[1828]: Closed GnuPG cryptographic agent and passphrase cache (restricted).
Sep 21 07:33:00 w00wmi systemd[1828]: Closed GnuPG cryptographic agent (ssh-agent emulation).
Sep 21 07:33:00 w00wmi systemd[1828]: Closed GnuPG cryptographic agent and passphrase cache.
Sep 21 07:33:00 w00wmi systemd[1828]: Closed PipeWire Multimedia System Socket.
Sep 21 07:33:00 w00wmi systemd[1828]: Closed debconf communication socket.
Sep 21 07:33:00 w00wmi systemd[1828]: Closed REST API socket for snapd user session agent.
Sep 21 07:33:00 w00wmi systemd[1828]: Removed slice User Application Slice.
Sep 21 07:33:00 w00wmi systemd[1828]: Reached target Shutdown.
Sep 21 07:33:00 w00wmi systemd[1828]: Finished Exit the Session.
Sep 21 07:33:00 w00wmi systemd[1828]: Reached target Exit the Session.
Sep 21 07:33:00 w00wmi systemd[1]: user@1000.service: Deactivated successfully.
Sep 21 07:33:00 w00wmi systemd[1]: user@1000.service: Deactivated successfully.
Sep 21 07:33:00 w00wmi systemd[1]: Stopped User Manager for UID 1000.
Sep 21 07:33:00 w00wmi systemd[1]: Stopping User Runtime Directory /run/user/1000...
Sep 21 07:33:00 w00wmi systemd[1]: run-user-1000.mount: Deactivated successfully.
Sep 21 07:33:00 w00wmi systemd[1]: user-runtime-dir@1000.service: Deactivated successfully.
Sep 21 07:33:00 w00wmi systemd[1]: Stopped User Runtime Directory /run/user/1000.
Sep 21 07:33:00 w00wmi systemd[1]: Removed slice User Slice of UID 1000.
After this, when running sudo systemctl restart user@1000
once it is never stopped again until I restart WSL. I am not sure when exactly it started in my case, but it worked before.
WSL version: 1.2.5.0
Kernel version: 5.15.90.1
WSLg version: 1.0.51
MSRDC version: 1.2.3770
Direct3D version: 1.608.2-61064218
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.19045.3448
Just found another interesting article: https://serverfault.com/questions/1139283/systemd-stops-user-manager-and-kills-all-user-processes
In my case I am running Ubuntu 22.04. After enabling lingering for my user it does not seem to be stopped anymore.
So my workaround: loginctl enable-linger 1000
sudo systemctl restart user@1000
This solved to me a
Failed to connect to bus: No such file or directory
issue. UsingguiApplications=false
here. Thanks @higordearaujo-bsy! 👍
For anyone running WSL2 Ubuntu 22.04 using systemd and encountering this issue, I found a very simple workaround.
sudo loginctl enable-linger $USER
./etc/wsl.conf
.wsl --shutdown
./etc/wsl.conf
.Note that I had enabled systemd while running 20.04 and it was working fine. This issue only surfaced when I did an in-place upgrade to 22.04. I'll note that before I figured this out, running sudo systemctl restart user@1000
did fix it, but I had to run it every time I restarted the WSL VM. However, enabling linger alone was having no effect.
I guess "Have you tried turning it off and on again?" never stops being sage advice.
Just found another interesting article: https://serverfault.com/questions/1139283/systemd-stops-user-manager-and-kills-all-user-processes
In my case I am running Ubuntu 22.04. After enabling lingering for my user it does not seem to be stopped anymore.
So my workaround:
loginctl enable-linger 1000
For me this started occurring after disabling GUI application support via the WSL Settings app. Enabling linger for the user seems to have resolved the issue.
Edit: I also disabled Hyper-V Firewall at the same time, but based on the comments in this thread, I figure that's unrelated.
Also see https://github.com/microsoft/WSL/issues/8879 for more systemd issues
Windows Version
Microsoft Windows [Version 10.0.22621.1778]
WSL Version
1.3.10.0
Are you using WSL 1 or WSL 2?
- [x] WSL 2
- [ ] WSL 1
Kernel Version
5.15.90.2-microsoft-standard-WSL2
Distro Version
Ubuntu 22.04
Other Software
No response
Repro Steps
Make sure
guiApplications=true
in wslconfig.
- Open WSL from Windows Terminal.
- Run
systemctl --user status
- Or Run
ls /run/user/1000
Expected Behavior
Excepts systemd user session to be present and systemctl --user to be able connect to it.
This kinda relates to #8842 , but not the same issue. This not the race condition issue, rather WSL login is nuking bus and other sockets in /run/user/1000 directory. Check the attached video demonstration.
If we follow the Following steps:
- In Powershell, wsl --shutdown few times to be sure.
wsl -u root -e /bin/bash
this makes wsl login to root. So wsl doesn't touch 1000 user (named gittu). this user has linger enabled, so systemd naturally creates the user session.- Here logged in via root,
ls /run/user/1000
shows proper bus, systemd etc.. sockets created.- Now open wsl to user gittu (which is default user) by opening a new tab in Windows Terminal.
- Observe that
[ 29.574698] WSL (2): Creating login session for gittu
line appears in dmesg output, confirming that WSL indeed created a user session for gittu, nuking previously good user session created by systemd.- Now output of
ls /run/user/1000/
doesn't have bus, systemd etc.. sockets.Now If I disable wslg, so guiApplications=false. The issue is solved, wsl doesn't nuke contents of /run/user/1000. So from this observation my conclusion is wslg is nuking the contents of /run/user/1000, and only manually putting wslg's files there.
Screenshot.2023-06-16.00.24.23.mp4
Actual Behavior
↪ ~ ➤ systemctl --user status Failed to connect to bus: No such file or directory ↪ ~ ➤ ls /run/user/1000 -l total 0 drwx------ 3 gittu gittu 60 Jun 16 00:24 dbus-1 drwx------ 2 gittu gittu 80 Jun 16 00:24 pulse lrwxrwxrwx 1 root root 31 Jun 16 00:25 wayland-0 -> /mnt/wslg/runtime-dir/wayland-0 -rw-rw---- 1 gittu gittu 0 Jun 16 00:24 wayland-0.lock
Diagnostic Logs
No response
Hi My response is not related to your problem... But I got a problem running one of the scripts cuz I'm totally noob
How can I get root login access in windows! ![Uploading SharedScreenshot.jpg…]()
I've just run across this myself in a newly installed Ubuntu 22.04 setup. It's a pretty big out-of-the-box flaw for anyone who actually wants to use user systemd. Is anyone working on it on the WSL team?
Workaround for WSL2 Ubuntu 22.04 Systemd Issue
For anyone running WSL2 Ubuntu 22.04 using systemd and encountering this issue, I found a very simple workaround.
1. `sudo loginctl enable-linger $USER`. 2. Disable systemd in `/etc/wsl.conf`. 3. Restart WSL using `wsl --shutdown`. 4. Re-enable systemd in `/etc/wsl.conf`. 5. Restart WSL again.
Note that I had enabled systemd while running 20.04 and it was working fine. This issue only surfaced when I did an in-place upgrade to 22.04. I'll note that before I figured this out, running
sudo systemctl restart user@1000
did fix it, but I had to run it every time I restarted the WSL VM. However, enabling linger alone was having no effect.I guess "Have you tried turning it off and on again?" never stops being sage advice.
sudo loginctl enable-linger $USER
works for me, also works after restart wsl
Windows Version
Microsoft Windows [Version 10.0.22621.1778]
WSL Version
1.3.10.0
Are you using WSL 1 or WSL 2?
Kernel Version
5.15.90.2-microsoft-standard-WSL2
Distro Version
Ubuntu 22.04
Other Software
No response
Repro Steps
Make sure
guiApplications=true
in wslconfig.systemctl --user status
ls /run/user/1000
Expected Behavior
Excepts systemd user session to be present and systemctl --user to be able connect to it.
This kinda relates to #8842 , but not the same issue. This not the race condition issue, rather WSL login is nuking bus and other sockets in /run/user/1000 directory. Check the attached video demonstration.
If we follow the Following steps:
wsl -u root -e /bin/bash
this makes wsl login to root. So wsl doesn't touch 1000 user (named gittu). this user has linger enabled, so systemd naturally creates the user session.ls /run/user/1000
shows proper bus, systemd etc.. sockets created.[ 29.574698] WSL (2): Creating login session for gittu
line appears in dmesg output, confirming that WSL indeed created a user session for gittu, nuking previously good user session created by systemd.ls /run/user/1000/
doesn't have bus, systemd etc.. sockets.Now If I disable wslg, so guiApplications=false. The issue is solved, wsl doesn't nuke contents of /run/user/1000. So from this observation my conclusion is wslg is nuking the contents of /run/user/1000, and only manually putting wslg's files there.
https://github.com/microsoft/WSL/assets/1235888/d32511fa-967f-4337-8da8-f08ea9468856
Actual Behavior
Diagnostic Logs
No response