Closed badjames closed 7 months ago
That is definitely not normal. I can only imagine it occurring if you are manually tunneling the VNC connection through SSH. For instance, if you had set up PuTTY to forward port 5903 from the TurboVNC host to a local port on the client, and if you are connecting to that local port using the TurboVNC Viewer, then it is expected that the VNC connection will drop when the SSH connection drops.
As a workaround, you can use the TurboVNC Session Manager instead. On the TurboVNC host, copy /etc/turbovncserver.conf to ~/.vnc, add $wm = "2d";
to ~/.vnc/turbovncserver.conf, then you can simply type the TurboVNC host name (without a port or a display name) into the TurboVNC Viewer. That will cause the TurboVNC Viewer to automatically use its built-in SSH client to connect to the host and either start a new TurboVNC session or present a dialog that allows you to select an existing TurboVNC session running under your user account. The TurboVNC Viewer's built-in SSH client supports password-less authentication with Pageant (PuTTY's private key agent.)
Still, though, it would be nice to understand why the issue is occurring. If you aren't tunneling the VNC connection through SSH, then there should be no dependency between the TurboVNC session and the SSH session in which it was started-- at least not with the default TurboVNC Server settings.
Thanks for the quick reply!
I tried what you suggested, but no joy. When I attempt to connect with just the server name specified, it fails with "Failed to connect to server".
One question that immediately springs to mind is how does the Windows TurboVNC client know the user/pass to SSH into the server?
Edit: I've just updated to the latest VNC Viewer 3.0.91 and it does give an option for passing a user name to use the TurboVNC Session Manager. I pass in:
user@host
But then a dialog appears for the password with the Windows username shown (not the user I specified) and is not editable.
In the above image, the Windows user name is "James" but the VNC server user name is different. This is what I entered in the main dialog:
There were several bugs with the handling of SSH usernames in the TurboVNC 3.1 beta Viewer. Try installing the 3.1 Post-Beta pre-release viewer (https://turbovnc.org/DeveloperInfo/PreReleases) on the Windows client.
Magic. The new version Session Manager works perfectly, thanks.
If you want to diagnose the issue with standard SSH & VNC disconnecting, I'd be happy to run any further tests...
Edit: spoke too soon :-(
A similar issue seems to exist for the Session Manager. I can start sessions fine, but when I close the TurboVNC window for the session, the session is terminated and doesn't appear in the list when I restart the Session Manager
OK, I think I've got to the bottom of it, just not managed to fix it yet.
Apparently there was a change added to systemd v230 in RHEL 8 (RHEL 7 uses 219) which causes it to kill all processes of a login session when it exits:
https://serverfault.com/a/463375
It should be fixable by adding "KillUserProcesses=no" to /etc/systemd/logind.conf.
This didn't seem to work for me, but I may need a reboot. This will have to wait for a few hours as I have a user actively on the machine.
OK, as a short term fix, the following works:
systemd-run --user -r /opt/TurboVNC/bin/vncserver -wm 2d :3
This will launch vncserver (silently, the output is lost) and it will remain running after the SSH client is exited.
I have to admit I'm a novice when it comes to systemd, so I don't fully understand how this works.
systemd is the bane of my existence, frankly. It's a horribly designed opaque mess that is antithetical to the principles of open source operating systems and causes untold numbers of issues for developers in my line of work. In order to fully comply with systemd, TigerVNC had to eliminate the vncserver script, require that all VNC sessions be started as root, limit it to one simultaneous desktop session (including VNC sessions and local logins) per user, and require a fixed display number for each user. It was, as you can imagine, an unpopular move that steered a lot of people toward TurboVNC (including all FreeBSD users, since FreeBSD doesn't even support systemd.) I have thus far refused to comply and have had to figure out ways of working around systemd in order to retain the ability to run multiple simultaneous TurboVNC sessions and allow ordinary users to start them. I guess this is going to require yet another workaround. KillUserProcesses
by default, but breaking decades-old assumptions about how SSH works seems like a really bad idea to me.
It will be Monday or Tuesday before I can look into this in detail. I'll keep you posted.
Thanks for that, and I agree with your opinion on systemd. It is great that you've gone to lengths to spare us from its pain :-)
As a final update, I'll just add that I tried setting KillUserProcesses=no but this did not have any effect, even after a reboot.
Sorry for the two-week delay. libjpeg-turbo distractions. :( Oddly, my Rocky 8.8 box doesn't experience this. I verified that it is running systemd v239. I started a session with the TurboVNC Session Manager and another session manually from an SSH shell. I then exited the SSH shell, closed the viewer window, and re-opened the TurboVNC Session Manager connection. Both sessions were still running.
$ grep KillUserProcesses /etc/systemd/logind.conf
#KillUserProcesses=no
So color me confused. If I change the value of KillUserProcesses
and reboot, I definitely do observe the issue. What could be different about our systems?
I don't observe the issue on Fedora 38 or Rocky 9.2 either. They both behave identically to my Rocky 8.8 machine, i.e. behaving as if KillUserProcesses=no
is the default.
I will have a further play at my end and maybe try an install on a different machine. Unfortunately, the machine in question has an active user, so I can't really hop on until the end of the day. I will let you know what I find out.
Apparently the change was proposed in Fedora, but this Bugzilla issue shows that the change was ultimately abandoned.
This thread was also enlightening (not to mention cathartic) and suggests that the upstream change to enable KillUserProcesses
by default was very controversial, and most downstream distros did not adopt it.
I look forward to your findings. If nothing else, it would probably be worthwhile to include a short blurb in the TurboVNC User's Guide that says to disable KillUserProcesses
if it isn't already disabled.
Thanks for further investigating.
From my side, I setup a new machine (VM this time), performed no configuration other than installing TurboVNC, and it works fine - exactly how you described. It only breaks when I manually set KillUserProcesses=true in logind.conf, and the default setting has it commented out.
So it seems specific to the physical PC I setup previously. I will go through some of the additional configuration steps I performed on it and see if the issue appears.
Keep you posted - although it might be tomorrow or later.
I'm going to go ahead and close this as can't reproduce, but feel free to add comments as you find out more. I am also proceeding with the TurboVNC 3.1 release.
Looks like it is the csh (tcsh) shell that is causing the termination on logout.
If the user running vncserver is using csh (set in /etc/passwd), logging out kills any running vncservers. Using the default "bash" shell, everything is fine. This is also new to Redhat (Rocky) 8, as we were running CentOS 7 with csh and never experienced the issue.
I will keep digging...
Interesting. I'll try to reproduce that on Monday. The TurboVNC Session Manager specifically invokes bash
on the remote machine in order to run the vncserver script, so I'm curious as to why the issue still exists with the Session Manager. Maybe SSH is still launching the session with csh, and the remote command has the effect of running bash inside of csh.
I did some more digging and got a little further. If I look at the process tree created when the Session Manager launches vncserver, I can see csh is invoked:
temp 95897 1 0 10:37 ? 00:00:00 sh -c (/opt/TurboVNC/bin/xstartup.turbovnc; /opt/TurboVNC/bin/vncserver -kill :1) >> '/ temp 95898 95897 0 10:37 ? 00:00:00 \_ -/bin/csh -c gnome-session temp 95997 95898 3 10:37 ? 00:00:00 \_ /usr/libexec/gnome-session-binary
The csh invocation comes from /opt/X11/xinit/Xsession (itself invoked from xstartup.turbovnc) at line 60:
57 gnome|gnome-session) 58 # lack of SSH_AGENT is intentional, see #441123. though 59 # the whole thing should really happen in xinitrc.d anyway. 60 exec -l $SHELL -c gnome-session 61 exec /bin/sh -c "exec -l $SHELL -c \"gnome-session\"" 62 ;;
I have fixed things fairly trivially at my end by adding:
export SHELL=bash
At the beginning of /opt/TurboVNC/bin/xstartup.turbovnc
I'm happy applying this fix myself to any RHEL8 installs, although you might want to dig further if you think others might be affected.
I will investigate modifying xstartup.turbovnc, because it is really intended to be executed with Bash anyhow.
The minimally intrusive patch is:
--- a/unix/xstartup.turbovnc
+++ b/unix/xstartup.turbovnc
@@ -193,6 +193,14 @@ if [ "$TVNC_WM" != "" ]; then
# Start the window manager.
if [ "$XSESSION" != "" ]; then
echo xstartup.turbovnc: Executing $TVNC_SSHAGENT $TVNC_VGLRUN $XSESSION \"$TVNC_WM\"
+ case $SHELL in
+ *bash)
+ ;;
+ *)
+ SHELL=bash
+ export SHELL
+ ;;
+ esac
exec $TVNC_SSHAGENT $TVNC_VGLRUN $XSESSION "$TVNC_WM"
else
echo xstartup.turbovnc: Executing $TVNC_SSHAGENT $TVNC_VGLRUN $TVNC_WM
However, in my testing, the issue does not occur with Ubuntu or even with Rocky 9, so it appears to be specific to RHEL 8 and workalike distributions. I am unwilling to modify the default xstartup.turbovnc until I understand why the issue occurs and why it's specific to RHEL 8. If, for instance, there is a default csh setting that causes it to terminate all background processes when the shell exits, then it would be better to document that setting than to override the intended functionality of the Xsession script.
A couple of other minor observations I've noted. Firstly, csh doesn't simply kill processes on exit. If I background an application in csh and exit the session, the application is left running.
The other thing worth mentioning is that there are a lot of "connection to bus lost" errors in /var/log/messages when exiting via csh which don't appear when exiting bash:
Nov 21 11:18:31 localhost systemd-logind[821]: Session 17 logged out. Waiting for processes to exit. Nov 21 11:18:31 localhost org.gtk.vfs.Daemon[37521]: A connection to the bus can't be made Nov 21 11:18:31 localhost journal[37836]: lost session service Nov 21 11:18:31 localhost org.gnome.Shell.CalendarServer[37611]: gnome-shell-calendar-server[37611]: Lost (or failed to acquire) the name org.gnome.Shell.CalendarServer - exiting Nov 21 11:18:31 localhost gnome-session-binary[37513]: WARNING: Lost name on bus: org.gnome.SessionManager Nov 21 11:18:31 localhost gnome-session[37513]: gnome-session-binary[37513]: WARNING: Lost name on bus: org.gnome.SessionManager Nov 21 11:18:31 localhost journal[37861]: Owner of volume monitor org.gtk.vfs.UDisks2VolumeMonitor disconnected from the bus; removing drives/volumes/mounts Nov 21 11:18:31 localhost org.freedesktop.Tracker1[37847]: Received signal:15->'Terminated' Nov 21 11:18:31 localhost org.freedesktop.Tracker1[37847]: OK Nov 21 11:18:31 localhost tracker-miner-fs.desktop[37861]: Received signal:15->'Terminated' Nov 21 11:18:31 localhost tracker-miner-fs.desktop[37861]: OK Nov 21 11:18:31 localhost journal[37861]: Owner of volume monitor org.gtk.vfs.MTPVolumeMonitor disconnected from the bus; removing drives/volumes/mounts Nov 21 11:18:31 localhost journal[37861]: Owner of volume monitor org.gtk.vfs.GPhoto2VolumeMonitor disconnected from the bus; removing drives/volumes/mounts Nov 21 11:18:31 localhost journal[37861]: Owner of volume monitor org.gtk.vfs.GoaVolumeMonitor disconnected from the bus; removing drives/volumes/mounts Nov 21 11:18:31 localhost journal[37861]: Owner of volume monitor org.gtk.vfs.AfcVolumeMonitor disconnected from the bus; removing drives/volumes/mounts Nov 21 11:18:31 localhost journal[37861]: Error while sending AddMatch() message: The connection is closed Nov 21 11:18:31 localhost journal[37861]: Error while sending AddMatch() message: The connection is closed Nov 21 11:18:31 localhost journal[37861]: Error while sending AddMatch() message: The connection is closed Nov 21 11:18:31 localhost cupsd[873]: REQUEST localhost - - "POST / HTTP/1.1" 200 152 Cancel-Subscription successful-ok Nov 21 11:18:31 localhost cupsd[873]: REQUEST localhost - - "POST / HTTP/1.1" 200 152 Cancel-Subscription client-error-not-found Nov 21 11:18:31 localhost journal[37680]: Source ID 3 was not found when attempting to remove it Nov 21 11:18:31 localhost journal[37583]: Lost or failed to acquire name org.gnome.Mutter.ScreenCast Nov 21 11:18:31 localhost journal[37583]: Lost or failed to acquire name org.gnome.Mutter.RemoteDesktop Nov 21 11:18:31 localhost gnome-session[38118]: Unable to init server: Could not connect: Connection refused Nov 21 11:18:31 localhost journal[38118]: Cannot open display:
Not sure that really helps, but if you are still investigating it might be useful.
I've also tried prefixing /opt/TurboVNC/bin/vncserver
with nohup
and /usr/bin/nohup
when starting a TurboVNC session. No effect. Explicitly specifying the WM and adding :noxsession
to the name (for example, passing -wm gnome:noxsession
to /opt/TurboVNC/bin/vncserver
or adding $wm = 'gnome:noxsession';
to /etc/turbovncserver.conf or ~/.vnc/turbovncserver.conf) works, but I haven't thoroughly tested or studied the ramifications of that. (Among other things, starting the WM without using the Xsession script will prevent any of the scripts in /etc/xinit/xinitrc.d from being executed, and it will prevent /etc/X11/Xresources and /etc/X11/Xmodmap from being loaded.)
If you want to avoid directly modifying /opt/TurboVNC/bin/xstartup.turbovnc (which would not survive a TurboVNC upgrade), then you can create a custom script (e.g. /etc/xstartup.turbovnc.local) with the following contents:
#!/bin/sh
case $SHELL in
*bash)
;;
*)
SHELL=bash
export SHELL
;;
esac
exec /opt/TurboVNC/bin/xstartup.turbovnc ${1+"$@"}
and add $xstartup = '/etc/xstartup.turbovnc.local';
to /etc/turbovncserver.conf.
I've just setup TurboVNC 3.0.91-20230818 on a new Rocky 8.8 (RHEL 8) installation.
It is all working perfectly with one exception - the TurboVNC session is killed when I exit the SSH session I started it from. More specifically:
I log into the machine from Windows using PuTTY and start a new vncserver session with:
/opt/TurboVNC/bin/vncserver -wm 2d :3
The server starts correctly and I can happily login using TurboVNC on Windows.
However, when I exit the SSH session (by typing "exit" into the PuTTY window), TurboVNC on Windows exits and the vncserver session is killed.
I've not had this on previous TurboVNC installations, so I'm not sure if it is a Rocky/RHEL 8 issue or not.