Install request: Xpra - Githubissues

heatherkellyucl commented 6 years ago

IN:03022221 For proof of concept right now.

Xvnc can't find the xkb rules (apparently you can tell it where they are at build time if you build from source) and users could connect to other people's sessions on the same box if they had the details, so not ideal. Xpra may work better.

https://xpra.org/ https://xpra.org/trac/wiki/Building https://xpra.org/trac/wiki/Dependencies

heatherkellyucl commented 6 years ago

Seeing how gallium does now.

heatherkellyucl commented 6 years ago

Huh, part of the make install failed:

Making install in compiler
make[3]: Entering directory `/lustre/shared/ucl/apps/xorg/full/source/2018-07-06/mesa/mesa-18.1.4/src/compiler'
  GEN      spirv/spirv_info.c
Traceback (most recent call last):
  File "./spirv/spirv_info_c.py", line 29, in <module>
    from mako.template import Template
ImportError: No module named mako.template
make[3]: *** [spirv/spirv_info.c] Error 1
make[3]: Leaving directory `/lustre/shared/ucl/apps/xorg/full/source/2018-07-06/mesa/mesa-18.1.4/src/compiler'
make[2]: *** [install-recursive] Error 1
make[2]: Leaving directory `/lustre/shared/ucl/apps/xorg/full/source/2018-07-06/mesa/mesa-18.1.4/src'
make[1]: *** [install] Error 2

heatherkellyucl commented 6 years ago

...what? I just installed mako for both pythons. And configure did not fail there before.

checking if module mako in python is installed... No module named mako
no
configure: error: Python mako module v0.8.0 or higher not found
build.sh: "./configure" failed on mesa/mesa
build.sh: error processing:  "mesa/mesa"

https://bugs.freedesktop.org/show_bug.cgi?id=95338 mesa only wants python2 mako. I was building with python3/recommended. But if you have no mako at all you don't get an error.

heatherkellyucl commented 6 years ago

Switched to python2/recommended. Note I do not know if anything in the first set of X modules was using python3, so should rerun the whole thing if this does in fact work to the end.

heatherkellyucl commented 6 years ago

Mesa built successfully, doing modfile3.

heatherkellyucl commented 6 years ago

modfile3 finished successfully, modfile4 stopped on driver/xf86-input-evdev:

checking for LIBEVDEV... no   
configure: error: Package requirements (libevdev >= 0.4) were not met:

No package 'libevdev' found   

Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively, you may set the environment variables LIBEVDEV_CFLAGS
and LIBEVDEV_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.
build.sh: "./configure" failed on driver/xf86-input-evdev
build.sh: error processing:  "driver/xf86-input-evdev"

heatherkellyucl commented 6 years ago

Annnnd out of time. (It should try to build libevdev if it had a tar file for it).

It got this far:

less resumefile4 
PASS: xserver
PASS: libevdev
PASS: libinput
FAIL: driver/xf86-input-evdev

Note that libevdev and libinput were both skipped. modfile4 contained:

xserver
libevdev
libinput
driver/xf86-input-evdev
driver/xf86-input-joystick
driver/xf86-input-libinput
driver/xf86-input-vmmouse
driver/xf86-input-keyboard
driver/xf86-input-mouse
driver/xf86-input-synaptics
driver/xf86-input-void
driver/xf86-video-sisusb
driver/xf86-video-sunffb
driver/xf86-video-v4l
driver/xf86-video-xgixp
driver/xf86-video-i740
driver/xf86-video-intel
driver/xf86-video-amdgpu
driver/xf86-video-apm
driver/xf86-video-ark
driver/xf86-video-ast
driver/xf86-video-ati
driver/xf86-video-chips
driver/xf86-video-cirrus
driver/xf86-video-dummy
driver/xf86-video-fbdev
driver/xf86-video-glint
driver/xf86-video-i128
driver/xf86-video-mach64
driver/xf86-video-mga
driver/xf86-video-neomagic
driver/xf86-video-nested
driver/xf86-video-nv
driver/xf86-video-rendition
driver/xf86-video-r128
driver/xf86-video-s3
driver/xf86-video-s3virge
driver/xf86-video-savage
driver/xf86-video-siliconmotion
driver/xf86-video-sis
driver/xf86-video-tdfx
driver/xf86-video-tga
driver/xf86-video-trident
driver/xf86-video-tseng
driver/xf86-video-vesa
driver/xf86-video-vmware
driver/xf86-video-voodoo
data/cursors
font/encodings
font/adobe-100dpi
font/adobe-75dpi
font/adobe-utopia-100dpi
font/adobe-utopia-75dpi
font/adobe-utopia-type1
font/arabic-misc
font/bh-100dpi
font/bh-75dpi
font/bh-lucidatypewriter-100dpi
font/bh-lucidatypewriter-75dpi
font/bh-ttf
font/bh-type1
font/bitstream-100dpi
font/bitstream-75dpi
font/bitstream-type1
font/cronyx-cyrillic
font/cursor-misc
font/daewoo-misc
font/dec-misc
font/ibm-type1
font/isas-misc
font/jis-misc
font/micro-misc
font/misc-cyrillic
font/misc-ethiopic
font/misc-meltho
font/misc-misc
font/mutt-misc
font/schumacher-misc
font/screen-cyrillic
font/sony-misc
font/sun-misc
font/winitzki-cyrillic
font/xfree86-type1
font/alias
util/cf
util/imake
util/gccmakedep
util/lndir
xkeyboard-config

Most of those drivers would be skipped.

ikirker commented 6 years ago

Erf, an attempt to use Singularity to containerise this may be more tricky than hoped. Various socket locations and filesystem nooks need to be made accessible to the container, and there may be group issues. We'll see, though.

ikirker commented 6 years ago

(So far it looks like you need to bind /run at least, and I'm not sure what's next -- it really wants to use systemd and dbus, and those are not going to play well with either running entirely in unprivileged userspace or through the container, I think.)

ikirker commented 6 years ago

Here's where I am right now:

Xpra wants to log to $XDG_RUNTIME_DIR I think, which is in /run, so bind that into the container (it wasn't bound by default).
Try to squash as many systemd and dbus-like options as I can find: no PulseAudio, no systemd-run.
The system proxy (whatever it does) wants write access to /run/xpra, which we don't have, so turn that off.
I got some GLX errors, so I specified an Xvfb command line that was the same as the default but removing the GLX extension.
I'm not sure what ssh-ing will do through the container boundary and such, so I was trying the TCP route instead, even though it's horribly insecure, just to get something that works I can move from.

singularity shell -B /run:/run docker://reto/x11-xpra
# Try to start the Xpra server
xpra start \
 --start=xterm \
 --systemd-run=no \
 --user=uccaiki \
 --xvfb="Xvfb +extension Composite -screen 0 5760x2560x24+32 -dpi 96 -noreset -auth $XAUTHORITY" \
--pulseaudio=no \
--start-via-proxy=no \
--bind-tcp=0.0.0.0:10000

# Try to attach to the server
xpra attach --opengl=no tcp:localhost:10000 --pulseaudio=no

Currently the server dies with some error about displayfd. There are some displayfd command-line options, but altering them doesn't seem to be helping at the moment and it's not obvious what the error message means.

ikirker commented 6 years ago

Okay, built my own Singularity recipe for Xpra instead of using the recommended one, since that was a Docker image.

It looks like I can get it to connect, but not to actually show anything. I'm going to add xterm into the container and try again tomorrow, since that should let me work out whether anything is happening.

Remember to bind /shared:/shared and /run:/run when you singularity exec.

ikirker commented 6 years ago

This... sort of works!

singularity shell -B /run -B /shared shub://UCL-RITS/rcps-shub-recipes:xpra
xpra start --systemd-run=no --user=$USER --start-via-proxy=no --start-on-connect=xterm --notifications=no
xpra attach --opengl=no --no-printing --notifications=no --webcam=no

ikirker commented 6 years ago

Current issues:

these are not short commands -- could dump them into scripts inside the container during %post
maybe some of the flags are not necessary -- I've already pruned some, but I could test more
despite my best efforts telling it to ignore pulseaudio and any other audio facility, it still prints an error about not being able to query a sound system
I'm not sure whether anything OpenGL-related will work

ikirker commented 6 years ago

Okay, I wrapped up the commands in /usr/bin/custom-xpra-server-start and /usr/bin/custom-xpra-attach inside the container.

And, OpenGL does work (testing with GaussView 6), but it's not pretty and it's not fast -- I guess it's using Mesa.

heatherkellyucl commented 6 years ago

I have tested this:

SSH in with X-forwarding on


[cceahke@login12 ~]$ singularity shell -B /run -B /shared shub://UCL-RITS/rcps-shub-recipes:xpra
Progress |===================================| 100.0% 
Singularity: Invoking an interactive shell within container...

Singularity UCL-RITS-rcps-shub-recipes-master-xpra.simg:~>

Then run the command to start:

Singularity UCL-RITS-rcps-shub-recipes-master-xpra.simg:~> /usr/bin/custom-xpra-server-start Singularity UCL-RITS-rcps-shub-recipes-master-xpra.simg:~> Entering daemon mode; any further errors will be reported to: /run/user/74700/xpra/S250264.log Actual display used: :0 Actual log file name is now: /run/user/74700/xpra/:0.log

You need to press enter there to get your prompt back. Then run the xpra attach command. There will be some errors.

Singularity UCL-RITS-rcps-shub-recipes-master-xpra.simg:~> /usr/bin/custom-xpra-attach

(process:250515): Gtk-WARNING **: 11:03:00.008: Locale not supported by C library. Using the fallback 'C' locale. 2018-10-01 11:03:00,811 Xpra gtk2 client version 2.1.3-r17247M 64-bit 2018-10-01 11:03:00,811 running on Linux Ubuntu 18.04 bionic Warning: failed to import GStreamer 1.x: Namespace Gst not available 2018-10-01 11:03:00,969 Error: failed to query sound subsystem: 2018-10-01 11:03:00,969 query did not return any data 2018-10-01 11:03:01,195 keyboard settings: rules=evdev, model=pc105, layout=gb 2018-10-01 11:03:01,273 desktop size is 3840x1200 with 1 screen: 2018-10-01 11:03:01,273 localhost:14.0 (1016x318 mm - DPI: 96x95) workarea: 3840x1173 2018-10-01 11:03:01,273 monitor 1 1920x1200 (518x324 mm - DPI: 94x94) 2018-10-01 11:03:01,273 monitor 2 1920x1200 at 1920x0 (518x324 mm - DPI: 94x94) 2018-10-01 11:03:01,814 enabled fast mmap transfers using 256MB shared memory area 2018-10-01 11:03:01,815 Xpra X11 server version 2.1.3-r17247 64-bit 2018-10-01 11:03:01,815 running on Linux Ubuntu 18.04 bionic 2018-10-01 11:03:01,815 enabled remote logging 2018-10-01 11:03:01,866 Attached to :0 (press Control-C to detach)


You also get an xterm. Ctrl-C in the Singularity terminal window will detach the session, and running the same three commands again will reattach it showing you whatever you put in the original xterm (and give you an additional new xterm that you can close).

ikirker commented 6 years ago

I think the daemon there is starting as a background process, so pressing Enter just gives you a new prompt: you could keep typing and stuff would happen but it would just be messy.

A significant caveat is that nothing will clean up left-over xpra servers automatically: because they're daemonised, exiting the containerised interactive shell won't do anything to them (pkill -u $USER xpra is probably a good thing to remember).

The -B options bind the paths given to the same paths within the container (you can bind them to different places using :, e.g. -B /blah:/foo would bind /blah outside to /foo inside. We need to bind /run for Xpra, and then we've also bound /shared to be able to access applications.

heatherkellyucl commented 6 years ago

/shared is bound, but module files aren't available in the xterm to start with. Need to check if sourcing defmods will do that or if that won't work.

Software they want to run:

We will be analysing fMRI data of about 0.5G each. Using both matlab and FSL(v. 5.0).
Commands with visualisation we would want to run are, for example: fsleyes, Feat (both from fsl), matlab.
We would be running other commands from the FSL package as well as python.
All output will be hopefully read and written from/to RDS.
If we run a jupyter notebook on the node would we be able to connect to it from our own computers?

Have explained that data would need to be explicitly copied from RDS first.

ikirker commented 6 years ago

I can tweak some /etc file setup in the container recipe if it makes things like modules work more smoothly.

ikirker commented 6 years ago

I seem to be having some Singularity Hub issues at the moment (https://github.com/singularityhub/singularityhub.github.io/issues/153) but in principle once they're resolved it should be possible to use modules straight from singularity shell without having to source anything extra.

If we run a jupyter notebook on the node would we be able to connect to it from our own computers?

This is going to be a problem for other reasons: everything in Myriad is behind the F5 and I don't think they'll pass anything except port 22 through. (See previous complaints X, Y, Z, A, B, C, etc.)

ikirker commented 6 years ago

Update: Singularity Hub issues are apparently Github's issues: webhooks have been disabled while they fix some stuff, which means updates to recipes won't be able to trigger rebuilds.

UCL-RITS / rcps-buildscripts

Install request: Xpra #186