Add template "vnccontainer" for containerized vnc servers

[georgiastuart]

vnccontainer template class

This pull request adds a new class to the batch_connect templates: vnccontainer.

Motivation

This feature was motivated by difficulties running the ANSYS software through Open OnDemand on a Rocky8 cluster at UT Dallas. Since Rocky8 is not an officially supported OS in the ANSYS specs, we had some display issues running ANSYS Workbench through the Open OnDemand desktop. As a solution, we decided to containerize a Centos8 machine--VNC server, xfce4, and all--and run that.

Also, we run a stateless cluster so installing desktops into the chroots really bloats the memory consumption. Containerizing everything solves that issue as well.

Implementation

vnccontainer.rb follows vnc.rb closely, but loads a container through apptainer/singularity instance start. All VNC and Websockify commands are run in the container instance. The container instance is shut down at the end of the session.

New options

vnccontainer adds several new options to batch_connect:

1. container_path ("vnc_container.sif") the path to the container with VNC
2. container_bindpath ("") paths to bind into the container with VNC
3. container_module ("singularity") the module that loads Singularity or Apptainer with Lmod. Supports versions (i.e. apptainer/1.10). If Singularity or Apptainer are installed at a system level (i.e., no module loaded to activate), set this to an empty string.
4. container_command ("singularity") the singularity or apptainer execution command
5. instance_name ("desktop") a name for the apptainer/singularity instance

vnccontainer sets the environmental variable INSTANCE_NAME for use in the app scripts.

Other considerations

This line in the ondemand repository also needs to be updated to

if session.script_type == "vnc" || session.script_type == "vncconnect"

in order for this class to work properly.

Testing

This class has been tested in two places:

1. An on-prem Rocky8 OpenHPC-based cluster running Open OnDemand.
2. An elastic Jetstream2 Rocky8 OpenHPC-based cluster running Open OnDemand.

This has ONLY been tested with Apptainer 1.10, but from investigation it should work with Singularity back to version 3.3 or so.

Possible improvements

Add a variable to enable the --nv flag for GPU access.

┆Issue is synchronized with this Asana task by Unito

[johrstrom]

Hi! First, thank you for the contribution.

Without looking at it in detail I'll have to figure out what we need from the user, what can be supplied through a module and in-between. Also, without looking at it, I wonder if instance_name is going to have collision, if say you have a 2nd job on the same machine.

Lastly - (again without looking at it) - we'll probably add podman and docker support in the same template.

None of any of this has to happen in this pull request, we can touch up and add stuff later, that's all just to say what I'm thinking. I'll leave specific comments/requests over the next few days.

[georgiastuart]

I wonder if instance_name is going to have collision, if say you have a 2nd job on the same machine.

This could certainly happen. I could add a check for existing instances and add a string of random characters to prevent collision.

we'll probably add podman and docker support in the same template.

This would be more significant, but not too much of a headache. The way I see it working is building the exec string at the beginning and referencing that throughout, rather than writing the exec stuff every time. That's probably a better design anyway!

[johrstrom]

Hi, sorry for the delay on this. I started to look into it today, got side tracked and had some trouble building a sif. Next week I should be able to look into it more.

[georgiastuart]

• [x] Need to rename the file to go along with https://github.com/OSC/ood_core/pull/774/commits/7e88accd2f8d69f6727685697dca7b0b440e0819

[georgiastuart]

Lastly - (again without looking at it) - we'll probably add podman and docker support in the same template.

I can take a look at this soon, by the way.

Edit: leaving this for another release

[johrstrom]

There's no need to support podman or docker now. OSC has partial support for podman but I don't think we'll use it in this use case. I'll bet singtainer is just fine for 80% of users. Which is to say, I'm happy to pull this in with just singtainer support.

It would throw off container_path (there's no path), but we can likely just alias container and container_path so folks can use either.

[johrstrom]

Hi - sorry for the delay if you've been waiting on me. Are you waiting on anything from my side?

[georgiastuart]

@johrstrom nope! I just need to do the refactor but have had a travel-heavy September. Sorry for the delay!

[johrstrom]

Sorry for the delay!

No problem at all. I just wanted to be sure there's nothing pending from my side.

I was waiting for OnDemand 2.0.29 for Ubuntu patches, but we didn't get any. Now other things have come up. Which is to say, I'm happy to put this in 2.0.29 rather soon, but also just as happy to wait. I don't know if there'll be a 30, but if there is it'll be even further away.

There's no rush, but I think end of next week or the week after is likely the timeframe I'm looking at for 2.0.29.

[georgiastuart]

Great! I'll target the end of the week or early next week.

[georgiastuart]

Connected to https://github.com/OSC/ondemand/pull/2323

[georgiastuart]

@johrstrom It's ready for another round of review!

[johrstrom]

Oh shoot, one last thing. If you want to squash these commits into 1 so you control the commit message you can do so now. Otherwise I'm happy to squash it and write the commit message myself (of course, giving you credit for all of it).

[georgiastuart]

@johrstrom squashed!

[johrstrom]

Thanks again @georgiastuart!

[georgiastuart]

Awesome! Thanks @johrstrom

[ll4strw]

@georgiastuart Would you share the singularity definition file that you use for ANSYS please? Thanks in advance.

[johrstrom]

Here's what I was using for testing, though it's MATE and has nothing do to with ANSYS.

Bootstrap: docker

From: centos:7

%environment
    PATH=/opt/TurboVNC/bin:$PATH

%post   
    yum install -y epel-release
    yum groupinstall -y 'MATE Desktop'
    yum install -y python3-pip
    pip3 install ts
    yum install -y https://yum.osc.edu/ondemand/latest/compute/el7Server/x86_64/python-websockify-0.8.0-1.el7.noarch.rpm
    yum install -y https://yum.osc.edu/ondemand/latest/compute/el7Server/x86_64/turbovnc-2.2.3-1.el7.x86_64.rpm
    yum remove -y tigervnc-server python3-pip mate-power-manager
    yum clean all
    rm -rf /var/cache/yum/*

[ll4strw]

@johrstrom Many thanks anyway.

[georgiastuart]

@ll4strw Here you go. It's a mess of ANSYS requirements (some duplicated), but it works for the most part with the VNC_Container class provided by this PR. We don't install ANSYS into the container but rather mount the install directory in afterwards, so this is just an environment module.

bootstrap: docker
from: centos:8.3.2011

%environment
  export PATH="/opt/TurboVNC/bin:$PATH"
  export XFSM_VERBOSE=1

%post
  sed -i -e "s|mirrorlist=|#mirrorlist=|g" /etc/yum.repos.d/CentOS-*
  sed -i -e "s|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g" /etc/yum.repos.d/CentOS-*
  dnf -y install dnf-plugins-core
  dnf -y install epel-release
  dnf -y config-manager --set-enabled powertools
  dnf -y install strace ncurses-libs pciutils llvm-libs alsa-lib at-spi2-atk at-spi2-core atk avahi-libs bzip2-libs cairo cairo-gobject cups-libs dbus-libs expat fontconfig freetype fribidi gdk-pixbuf2 glib2 glibc glibc-devel gmp gnutls graphite2 gtk3 harfbuzz keyutils-libs krb5-libs libICE libSM libX11 libX11-xcb libXScrnSaver libXau libXcomposite libXcursor libXdamage libXext libXfixes libXft libXi libXinerama libXrandr libXrender libXtst libblkid libcap libcom_err libdatrie libdrm libepoxy libffi libgcc libgcrypt libglvnd-opengl libgpg-error libidn2 libmount libnsl libpng libselinux libstdc++ libtasn1 libthai libunistring libuuid libwayland-client libwayland-cursor libwayland-egl libwayland-server libxcb libxcrypt libxkbcommon lz4-libs mesa-libgbm nettle nspr nss nss-util openssh-clients openssl openssl-libs p11-kit pango pcre pcre2 pixman redhat-lsb-core systemd-libs xz-libs zlib
  dnf -y install aspell brotli enchant2 gstreamer1 gstreamer1-plugins-base harfbuzz-icu hyphen jbigkit-libs libXmu libXp libXt libXxf86vm libcurl-devel libgfortran libglvnd libglvnd-egl libglvnd-gles libglvnd-glx libgomp libibverbs libicu libjpeg-turbo libnotify libpng12 libpng15 libquadmath libsecret libsoup libtiff libtool-ltdl libuuid-devel libwebp libxml2 libxshmfence libxslt mesa-libGLU motif mpfr openjpeg2 openssl-devel orc pcre-devel perl-devel sqlite-libs ucx webkit2gtk3 webkit2gtk3-jsc woff2
  dnf -y install glibc.i686 mesa-libglapi
  dnf -y install aspell brotli enchant2 gstreamer1 gstreamer1-plugins-base harfbuzz-icu hyphen jbigkit-libs libXmu libXp libXt libXxf86vm libcurl-devel libgfortran libglvnd libglvnd-egl libglvnd-gles libglvnd-glx libgomp libibverbs libicu libjpeg-turbo libnotify libpng12 libpng15 libquadmath libsecret libsoup libtiff libtool-ltdl libuuid-devel libwebp libxml2 libxshmfence libxslt mesa-libGLU motif mpfr openjpeg2 openssl-devel orc pcre-devel perl-devel sqlite-libs ucx webkit2gtk3 webkit2gtk3-jsc woff2
  dnf -y install libICE.i686 libSM.i686 libX11.i686 libXau.i686 libXt.i686 libgcc.i686 libjpeg-turbo.i686 libstdc++.i686 libuuid.i686 libxcb.i686
  dnf -y install libnsl.i686 libnsl
  dnf -y install libXp.i686
  dnf -y group install "Development Tools"
  dnf -y install libnsl.i686
  dnf -y install libXp.i686
  dnf -y install alsa-lib at-spi2-atk at-spi2-core atk avahi-libs bzip2-libs cairo cairo-gobject cups-libs dbus-libs expat fontconfig freetype fribidi gdk-pixbuf2 glib2 glibc gmp gnutls graphite2 gtk3 harfbuzz keyutils-libs krb5-libs libICE libSM libX11 libX11-xcb libXScrnSaver libXau libXcomposite libXcursor libXdamage libXext libXfixes libXft libXi libXinerama libXrandr libXrender libXtst libblkid libcap libcom_err libdatrie libdrm libepoxy libffi libgcc libgcrypt libgpg-error libidn2 libmount libnsl libpng libselinux libstdc++ libtasn1 libthai libunistring libuuid libwayland-client libwayland-cursor libwayland-egl libwayland-server libxcb libxcrypt libxkbcommon lz4-libs mesa-libgbm nettle nspr nss nss-util openssl-libs p11-kit pango pcre pcre2 pixman redhat-lsb-core systemd-libs xz-libs zlib
  dnf -y install libICE.i686 libSM.i686 libX11.i686 libXau.i686 libXt.i686 libgcc.i686 libjpeg-turbo.i686 libstdc++.i686 libuuid.i686 libxcb.i686 glibc.i686
  dnf -y install aspell brotli enchant2 gstreamer1 gstreamer1-plugins-base harfbuzz-icu hyphen jbigkit-libs libXmu libXp libXt libXxf86vm libcurl-devel libgfortran libglvnd libglvnd-egl libglvnd-gles libglvnd-glx libgomp libibverbs libicu libjpeg-turbo libnotify libpng12 libpng15 libquadmath libsecret libsoup libtiff libtool-ltdl libuuid-devel libwebp libxml2 libxshmfence libxslt mesa-libGLU motif mpfr openjpeg2 openssl-devel orc pcre-devel perl-devel sqlite-libs ucx webkit2gtk3 webkit2gtk3-jsc woff2
  dnf -y install aspell audit-libs brotli compat-openssl10 cyrus-sasl-lib enchant2 gstreamer1 gstreamer1-plugins-base harfbuzz-icu hyphen jbigkit-libs libXdmcp libXmu libXp libXt libXxf86vm libcap-ng libcurl-devel libfontenc libgfortran libglvnd libglvnd-egl libglvnd-gles libglvnd-glx libgomp libibverbs libicu libjpeg-turbo libnotify libpng12 libpng15 libquadmath libsecret libsoup libtiff libtool-ltdl libuuid-devel libwebp libxkbcommon-x11 libxml2 libxshmfence libxslt mesa-libGLU motif mpfr ocl-icd-devel openjpeg2 openldap openssl-devel orc pam pciutils-libs pcre-devel perl-devel sqlite-libs tbb ucx webkit2gtk3 webkit2gtk3-jsc woff2
  dnf -y install aspell brotli enchant2 gstreamer1 gstreamer1-plugins-base harfbuzz-icu hyphen jbigkit-libs libXmu libXp libXt libXxf86vm libcurl-devel libgfortran libglvnd libglvnd-egl libglvnd-gles libglvnd-glx libgomp libibverbs libicu libjpeg-turbo libnotify libpng12 libpng15 libquadmath libsecret libsoup libtiff libtool-ltdl libuuid-devel libwebp libxml2 libxshmfence libxslt mesa-libGLU motif mpfr openjpeg2 openssl-devel orc pcre-devel perl-devel sqlite-libs ucx webkit2gtk3 webkit2gtk3-jsc woff2
  dnf -y install https://yum.osc.edu/ondemand/2.0/compute/el8/x86_64/turbovnc-2.2.5-1.el8.x86_64.rpm
  dnf -y install https://yum.osc.edu/ondemand/2.0/compute/el8/x86_64/turbovnc-debuginfo-2.2.5-1.el8.x86_64.rpm
  dnf -y install https://yum.osc.edu/ondemand/2.0/compute/el8/x86_64/python-websockify-0.8.0-1.el8.noarch.rpm
  dnf -y install xfce4-clipman-plugin xfce4-session xfce4-settings xfce4-terminal xfdesktop Thunar firefox mousepad libglvnd-glx xorg-x11-xauth xorg-x11-xkb-utils libxfce4ui
  dnf -y install openssh-clients openssh-server
  dnf -y install MUMPS-devel compat-wxGTK3-gtk2 gstreamer1 gstreamer1-plugins-base gtk2 jbigkit-libs libXmu libXp libXt libXxf86vm libcurl-devel libgfortran libglvnd libglvnd-glx libgomp libibverbs libjpeg-turbo libjpeg-turbo-devel libnotify libpng-devel libpng12 libpng15 libquadmath libxshmfence libtiff libtiff-devel libtool-ltdl libuuid-devel libxml2 libxslt mesa-libGLU motif mpfr numactl-libs ocl-icd orc pcre-devel perl-devel qhull-devel tbb ucx wxBase3 zlib-devel     
  dnf -y install mesa-libGL mesa-libGL-devel
  dnf -y install audit-libs compat-openssl10 cyrus-sasl-lib gstreamer1  gstreamer1-plugins-base hwloc-libs jbigkit-libs libXdmcp libXmu libXp libXt libXxf86vm libcap-ng libcurl-devel libfontenc libgfortran libglvnd libglvnd-egl libglvnd-glx libgomp libibverbs libjpeg-turbo libpng12 libpng15 libquadmath libtiff libtool-ltdl libuuid-devel libxkbcommon-x11 libxml2 libxshmfence libxslt mesa-libGLU motif mpfr ocl-icd ocl-icd-devel openldap orc  pam pciutils-libs pcre-devel perl-devel tbb ucx      
  dnf -y install jbigkit-libs libXmu libXp libXt libXxf86vm  libcurl-devel libgfortran libglvnd libglvnd-glx libgomp libibverbs libquadmath libtiff libtool-ltdl libuuid-devel libxml2 libxshmfence libxslt mesa-libGLU motif mpfr ocl-icd pcre-devel perl-devel tbb ucx 
  rm /opt/TurboVNC/etc/turbovncserver-security.conf
  rm /etc/ssh/ssh_config.d/05-redhat.conf

[ll4strw]

@georgiastuart Thank you.

[johrstrom]

This was released in version 2.0.29 today.

[georgiastuart]

Cool!! Thanks @johrstrom

[ll4strw]

Very nice indeed. Just one thing though, yesterday's announcement on the OpenonDemand list read [Open OnDemand] [Announcements] Open OnDemand 2.0.28 now available with body text

Open OnDemand 2.0.28 is now available. The biggest change is around our NodeJS dependency. 
NodeJS 12 has come to end of life for all platforms, so it’s no longer receiving security patches. 
So we had to upgrade to NodeJS 14 at this time.

And then it goes on with

See the instructions on upgrading from 2.0.28 to 2.0.29 here:
https://osc.github.io/ood-documentation/latest/release-notes/v2.0-release-notes.html#upgrading-to-v2-0-29

I guess there's been a mix up of version numbers here?

[johrstrom]

Thank you, yes there was a mistake - I've updated the discourse topic for the same.