Establishing a Dedicated Login Node for Interactive Computing and JupyterHub Integration

[ickc]

@DanielBThomas, @rwf14f, @afortiorama,

Introduction and Summary

Following our recent internal meeting, we are planning to deploy a JupyterHub instance, inspired by the model implemented at the National Energy Research Scientific Computing Center (NERSC). For more information, refer to NERSC's Jupyter Documentation and JupyterHub@NERSC.

Project Objective

Our primary goal is to establish a robust Jupyter environment, mirroring NERSC's implementation. Jupyter, an advanced computing platform, supports digital notebooks that combine executable code, equations, data visualization, interactive interfaces, and text. This deployment is particularly vital for enabling exploratory data analysis and visualization, especially with data stored at NERSC. This initiative gains added importance as the Simons Observatory collaborators plan to release Jupyter Notebooks for training, tutorials, and exploratory data analysis (EDA). It's essential that our infrastructure effectively supports these activities.

System Requirements

1. Hardware Configuration: We require a single physical node, robustly equipped with substantial CPUs, RAM, and disk space.
2. Home Directory Configuration:
   • We aim for the login node to also serve as the JupyterHub node, allowing users to maintain their HOME environment consistently, similar to their setup on vm77, whether accessing via SSH or JupyterHub.
   • This setup does not imply a merger of vm77 into this system; vm77 will continue functioning as a job submission node, providing redundancy.
   • A shared HOME environment between vm77 and the new node is not part of our current plan.
3. Intended Use of the Login Node:
   • The node is designated mainly for lightweight computation, interactive use, and visualization.
   • It will support running Jupyter Notebooks via JupyterHub and command-line tools, possibly with X11Forwarding.
   • Additionally, it's suitable for compilation tasks and running short, small-scale tests, potentially for continuous integration (CI) purposes.
4. Domain Assignment: A future consideration includes assigning a domain, such as jupyter.souk.ac.uk.

Login Process

• Our preferred method involves opening a specific port in the firewall, allowing UNIX username and password authentication. This approach might present security challenges, which we aim to address with Multi-Factor Authentication (MFA) or OAuth. We welcome suggestions regarding security requirements.
• As an alternative, SSH forwarding is an option, though it might be less convenient for users.

Maintenance and Security

• Our Data Centre software engineers will maintain the JupyterHub instance, in line with the JupyterHub Security Guidelines.
• We are prepared to initially deploy using SSH tunneling, considering the opening of a firewall port as a subsequent step.
• We will commence with a single node, distinct from worker nodes, meaning that job launches from Notebooks will not be possible. We plan to reevaluate this arrangement as future needs and feedback dictate.

[ickc]

Hi @rwf14f, what's your thoughts on this?