eWaterCycle / ewatercycle

Python package for running hydrological models
https://ewatercycle.readthedocs.io/en/latest/
Apache License 2.0
33 stars 5 forks source link

docs launching ewatercycle on mistral #289

Open jeromaerts opened 2 years ago

jeromaerts commented 2 years ago

For lack of a better place I will store my findings on how to run ewatercycle on Mistral in this issue with the aim that this will be part of the future docs.

On Mistral we can launch a Jupyter Lab server on a compute node using: https://jupyterhub.dkrz.de/hub/login. However, this server does not load Singularity as a default and it is not possible to load Singularity from the terminal or notebooks.

The solution: Create a shell script that loads Singularity and call this shell script when launching the kernel, see: https://jupyterhub.gitlab-pages.dkrz.de/jupyterhub-docs/posts/2021/2021-02-16_kernels_advanced.html.

The shell script: `#!/bin/bash

module purge module load singularity/3.6.1-gcc-9.1.0 . ~/mamba/bin/activate ewatercycle python -m ipykernel_launcher -f "$1" `

Run: chmod +x shell_script.sh

Modify: "~/.local/share/jupyter/kernels/ewatercycle/kernel.json" To:

{ "argv": [ "/path/to/file/shell_script.sh", "{connection_file}" ], "display_name": "ewatercycle", "language": "python" }

Peter9192 commented 2 years ago

Good one! Indeed, we have similar instructions for a custom ESMValTool kernel: https://github.com/ESMValGroup/ESMValTool-JupyterLab. Those instructions might also be useful for such documentation.

BSchilperoort commented 6 months ago

It would be nice to have some documentation on this, but we currently do not have access to Mistral and the steps you went through are outdated (e.g. Singularity is deprecated for eWaterCycle, we now use apptainer).

If anyone uses Mistral again, it would be nice to pick up this issue and add the info to the documentation.