Premium Offer

[achtsnits]

• should provide access to GPU for x hours based on subscription package
• should allow the headless execution of parametrizable notebooks for training and inference
• should provide generic parametrizable example notebooks

[achtsnits]

we did the setup for Juan as our "first premium user" now (note: we needed username, email as well as eotdl-token for the setup upfront!)

• parametrizable notebooks are located in ~/team-extra folder (note: Juan may decide to authorize additional users to be part of his team and can also work with data in this shared folder but no other non-authorized users have access to the files in this folder!)
• currently we only put one example parametrizable notebook in this folder upfront -> EOTDLDataset_Training.ipynb (note: Juan may add more for headless execution)
• Juan can invoke any notebook within the ~/team-extra folder via HTTP call ("headless execution"), a JupyterHub API Token (see https://jupyterhub.readthedocs.io/en/stable/howto/rest.html#create-an-api-token ) is necessary for that

example call:

curl -X POST https://hub.api.eotdl.com/services/eoxhub-gateway/eotdl/pygeoapi/processes/execute-notebook/execution \
     --header 'Content-Type: application/json' \
     -H "Authorization: Token <JupyterHub API Token>" \
     --data-raw '{"inputs": {
  "notebook": "team-extra/EOTDLDataset_Training.ipynb",
  "parameters_json": {"max_epochs": 2, "dataset_name": "EuroSAT-Q1-small"}
}}'

note the variants like max_epochs and dataset_name which are rendered in the notebook

(see papermill documentation https://papermill.readthedocs.io/en/latest/usage-parameterize.html)

now the specified notebook is executed on a GPU node -> this execution is fully tracked and all necessary info like the executed rendered notebook, logs and downloaded data are collected

the current example notebook still must be extended to create and store e.g. a model in case of training!

note: based on the JupyterHub API token from Juan the call is automatically routed to the correct endpoint dedicated to Juan!

[juansensio]

That looks great, thanks. I'll play with it and let you know how it goes. I'll update the notebook to include the final step to ingest the model to the EOTDL.