tclose commented 7 months ago

Metadata

Epic: Feature: Feature Release: Required knowledge: shallow or mid-level or deep

Description

Create documentation page on AIS developer website documenting how to set up CVMFS on k8s on NeuroDesk

Acceptance Criteria

[ ] 1. Webpage in Dev docs describing CVMFS setup
[ ] 2. Another AIS team member checks the docs and see whether they can follow them (e.g. @fxusyd )

Blocked by

[ ]

anzhao commented 6 months ago

Install the CernVM-FS client. On Ubuntu, you can install it by executing the following commands:

Configure the CVMFS. For the basic config, run cvmfs_config setup. This ensures that the file /etc/auto.master.d/cvmfs.autofs exists containing /cvmfs /etc/auto.cvmfs and that the autofs service is running. Reload the autofs service in order to apply an updated configuration. Once done, create the keys and configure the neurodesk.ardc.edu.au as below:

Test if the Neurodesk CVMFS repo is reachable or not (cvmfs_config stat -v neurodesk.ardc.edu.au).

anzhao commented 5 months ago

Setup the CVMFS and NeuroDesk

What is CVMFS

CVMFS stands for CernVM File System, which provides a scalable, reliable and low-maintenance software distribution service. CVMFS is implemented as a POSIX read-only file system in user space (a FUSE module). All the files are hosted on standard web servers and mounted in universal namespace /cvmfs. You can think CVMFS as a contnent distribution network for software. It makes the centrally managed and updated software repositories avaibale acorss resources. Once you have access a sofeware via CVMFS, it will be cached locally, thus providing runtime performancve comparable to locally installed versions.

How to install CVMFS

The CernVM-FS source code and binary packages are available from the CernVM website. As an example, on Ubuntu, you can install it by executing the following commands:

configure Neurodesk hosted on CVMFS

run "cvmfs_config setup" to do the initial setup and ensures that the file /etc/auto.master.d/cvmfs.autofs exists containing /cvmfs /etc/auto.cvmfs and that the autofs service is running. Reload the autofs service in order to apply an updated configuration. Once done, create the keys and configure the neurodesk.ardc.edu.au as below:

Test the overal cvmfs configurations:

First, test the overall cvmfs config setup by running "sudo cvmfs_config chksetup" as below:

Second, check if you can list the Neurodesk containers and modules:

Third, check the speed and nearest Neurodesk cvmfs host:

Setup the JupyterHub plugin:

Before we are able to launch the NeuroDesk, we must make sure that the JupyterHub is able to communicate with our XNAT. One critial step is to figure out the correct JupyterHub API URL shown as below:

In the meanwhile, remember to enable the Jupyter for the users.

In the Jupyter environment, please pick the Neurodesk Image you want to launch and in the following example, we pick the image "vnmd/neurodesktop:2024-01-12".

You can setup additional environment by click the above "New Jupyter Environment" button and the following window will pop up.

Allocate hardware to the jupyter users.
- You can create Hardware configurations that define the hardware resources that will be allocated to the single-user server when it is launched. Additional configuration options (such as cpu and memory limits) can be specified

Launch the Neurodesk:

After you have setup all the above steps, you can launch the Neurodesk by visiting any XNAT project and click the "start jupyter" in the right side as below. Remember to select the Neurodesk image and desired hardware size.