Motivation

Enhance the JupyterHub documentation to include comprehensive instructions for configuring the Rucio extension and Rucio Command Line Interface (CLI) in a JupyterHub environment, such as Z2JH, with a focus on the OpenID Connect (OIDC) authentication method and process. The documentation should be user-friendly and provide step-by-step guidance to ensure seamless setup and configuration of Rucio in a JupyterHub environment.

Modifications

rucio extension documentation for JupyterHub with OIDC authentication method, including
- Docker image for single user
- config.yaml for JupyterHub with necessary Python functions for OIDC tokens

There is some existing work that showcases the setup of the OIDC authentication method for Rucio in JupyterHub in the linked PRs.

There is also a demo repository hosted on my private GitHub account that could potentially be migrated to the Rucio organization or integrated into the https://github.com/rucio/k8s-tutorial repository.

In the end, the documentation should be hosted on the Rucio website as well.

Expected result

The expected outcome of this effort is comprehensive documentation that serves as a reference guide for setting up Rucio in a JupyterHub environment. This documentation should cover all the necessary configuration files, and Docker images, and provide clear instructions for a successful setup of Rucio with OIDC authentication in a JupyterHub environment.

Problems and tests

Currently, I'm testing the Rucio extension in download mode with rucio version 1.30.3 for local tests and 1.29.6 on JupyterHub. The auth server runs on version 1.29.6.

Token format

The decoded token jwt access_token looks like this:

{'wlcg.ver': '1.0', 'sub': '<hide>', 'aud': 'rucio', 'act': {'sub': '<hide>'}, 'nbf': 1681724949, 'scope': 'email openid profile', 'iss': 'https://iam-escape.cloud.cnaf.infn.it/', 'exp': 1681728549, 'iat': 1681724949, 'jti': '<hide>', 'client_id': '<hide>'}

A simple request to the Rucio whoami API with the token looks like this:

{"account_type": "SERVICE", "email": "domenic.gosein@cern.ch", "deleted_at": null, "updated_at": "2022-09-16T09:00:13", "status": "ACTIVE", "suspended_at": null, "created_at": "2022-09-16T09:00:13", "account": "dogosein"}

So the token seems to be valid and working.

Rucio Extension with token stored in an environment variable

When the above token is stored in an environment variable and as such specified in the config, the extension gets setup fine and DIDs can be listed, however when downloading a file ("Make Available") the following error is encountered:

Rucio Client did not succeed to get AuthN/Z URL from the Rucio Auth Server.                                    
This could be due to wrongly requested/configured scope, audience or issuer.
Process Process-1:
Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/opt/conda/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/conda/lib/python3.8/site-packages/rucio_jupyterlab/rucio/download.py", line 35, in start_download_target
    results = RucioFileDownloader.download(dest_folder, did)
  File "/opt/conda/lib/python3.8/site-packages/rucio_jupyterlab/rucio/download.py", line 62, in download
    client = Client()
  File "/opt/conda/lib/python3.8/site-packages/rucio/client/client.py", line 77, in __init__
    super(Client, self).__init__(**args)
  File "/opt/conda/lib/python3.8/site-packages/rucio/client/baseclient.py", line 282, in __init__
    self.__authenticate()
  File "/opt/conda/lib/python3.8/site-packages/rucio/client/baseclient.py", line 929, in __authenticate
    self.__get_token()
  File "/opt/conda/lib/python3.8/site-packages/rucio/client/baseclient.py", line 827, in __get_token
    raise CannotAuthenticate('OIDC authentication failed for account=%s' % self.account)
rucio.common.exception.CannotAuthenticate: Cannot authenticate.
Details: OIDC authentication failed for account=None

At the moment is it not quite clear how the account needs to be set.

Using the rucio CLI in the notebook

Using the OIDC token in the notebook with the rucio CLI for download does not work and throws the following error:

2023-04-17 08:38:48,473 DEBUG   logging.py      gfal.Default: downloading file from gsiftp://eulakeftp.cern.ch:2811/eos/eulake/tests/rucio_test/eulake_1/elena_test/c5/3a/test-file-rucio-2023-04-17-01.txt to /home/jovyan/elena_test/test-file-rucio-2023-04-17-01.txt.part
2023-04-17 08:38:48,524 DEBUG   downloadclient.py       The requested service is not available at the moment.
Details: An unknown exception occurred.
Details: Could not open source: globus_ftp_control: gss_init_sec_context failed GSS failure:  GSS Major Status: General failure GSS Minor Status Error Chain: globus_gsi_gssapi: Error with gss context globus_gsi_gssapi: Error with gss credential handle globus_credential: Valid credentials could not be found in any of the possible locations specified by the credential search order. Valid credentials could not be found in any of the possible locations specified by the credential search order. Attempt 1 globus_credential: Error reading host

Further tests

Further tests have shown that:

download of the same file with a different authentification method (e.g. X509) works fine in the extension as well as with the rucio CLI in JupyterLab Notebook
download on a local machine with a token generated by the rucio CLI works fine

This points to an issue with the token or missing configuration with the extension. Further debugging needs to be done to find the missing pieces.

rucio / jupyterlab-extension

JupyterHub OIDC authentication method and process #20