Example for Singularity and SLURM

[tornede]

Create an example folder containing

• Example experiment configuration
• Python example code
• Singularity recipe
• SLURM script for execution

[fmohr]

I have an example, are you interested?

[helegraf]

If you have an example ready then it would be helpful if you could share it. :)

[fmohr]

This is for using it with Python 3.8:

Example configuration is obsolete. Use your own ;-)

Same for python example code. In my case, the script is called runexperiment.py. Then:

Singularity Recipe

Bootstrap: docker
From: python:3.8-bullseye

%runscript
    exec echo "The runscript is the containers default runtime command!"

%files
cmake-3.19.3-Linux-x86_64.sh /

%environment

%labels
    Maintainer fmohr

%post
    # update container
    apt-get update

    # prepare for python 3.8
    apt-get -y install software-properties-common
    add-apt-repository ppa:deadsnakes/ppa

    apt-get install -y apt-utils

    #install packages
    apt-get -y install git

    # update pip
    echo "Updating pip\n------------------------------"
    LC_ALL=C.UTF-8 python3.8 -m pip install --upgrade pip

    # install cython, numpy and scipy
    echo "Now installing cython, numpy, and scipy\n------------------------------"
    LC_ALL=C.UTF-8 python3.8 -m pip install cython numpy scipy psutil

    # install tqdm
    LC_ALL=C.UTF-8 python3.8 -m pip install tqdm

    echo "Now installing scikit-learn and openml\n------------------------------"
    LC_ALL=C.UTF-8 python3.8 -m pip install scikit-learn
    LC_ALL=C.UTF-8 python3.8 -m pip install openml
    LC_ALL=C.UTF-8 python3.8 -m pip install py_experimenter
    LC_ALL=C.UTF-8 python3.8 -m pip install mysql-connector-python==8.0.29

    # install pebble, matplotlib, func_timeout
    LC_ALL=C.UTF-8 python3.8 -m pip install pebble matplotlib func_timeout psutil

    # update PANDAS
    LC_ALL=C.UTF-8 python3.8 -m pip install --upgrade pandas numpy

SLURM script

#!/bin/bash
#SBATCH -p normal
#SBATCH -A <your-group-name>             # the Slurm account to use (e.g. hpc-prf-dfhy)
#SBATCH --job-name=<name-of-job>         # create a short name for your job
#SBATCH --nodes=1                        # node count
#SBATCH --ntasks=1                       # total number of tasks across all nodes
#SBATCH --cpus-per-task=16               # cpu-cores per task (>1 if multi-threaded tasks)
#SBATCH --mem-per-cpu=12G                # memory per cpu-core (4G per cpu-core is default)
#SBATCH --time=09:55:00                  # total run time limit (HH:MM:SS)

module load system singularity
FOLDER=<absolute folder to your experiment code>

#SLURM_JOB_ID=test # this can be commented in if you want to run the script manually in the console prior to submitting

cd $FOLDER
echo "Working in $FOLDER"
echo "Job ID is: $SLURM_JOB_ID"
echo "Starting singularity."
singularity exec pc2.simg bash -c "python3.8 runexperiment.py $SLURM_JOB_ID"

Here, pc2.simg is the singularity image compiled with the above recipe, and the runexperiment.py has one parameter, which is used as the name of the experimenter. This way, you can track back the logs based on the name column in the result table.