Parametrized training templates

[juansensio]

Create parametrized training templates.

Requires generic Q1+ wrappers -> This is a class in PytorchEO called "EOTDLQ1Dataset" that accepts the name of the dataset as a parameter.

[juansensio]

Use cases:

• User goes to their workspace, select the template, change the code and execute.
• User click on train -> redirect to trianing UI -> input parameters -> execute. Parameters: dataset, resources (cpu/gpu), global hyperparameters (number of epochs, learning rate, batch size, ... but not really much else).

[juansensio]

@juansensio notebook examples with different datasets but for the same task to identify where the parameters should go (with the generic wrapper).

[achtsnits]

List view should show invocations separately per dataset and per user!

[juansensio]

A first iteration of the generic wrapper can be found here: https://github.com/earthpulse/pytorchEO/blob/main/examples/eotdl_wrapper.ipynb

It works for classification tasks and Sentinel 2 for the moment, we will improve as new Q1 datasets are ingested and the metadata specification is improved.

Use this to implement a first parametrized training template, using the EuroSAT-Q1-small as demo.

[juansensio]

Resuming this task @achtsnits @Schpidi, what is the status on this ?

[achtsnits]

we upgraded your user to the Premium offer so you can make use of the headless execution capabilities with parametrized notebooks -> see steps (and what we need for onboarding) described https://github.com/earthpulse/eotdl/issues/194#issuecomment-2163030318

the notebook EOTDLDataset_Training.ipynb used there is based on above https://github.com/earthpulse/pytorchEO/blob/main/examples/eotdl_wrapper.ipynb, so you can adapt/create further parametrized training as well as inference examples yourself and test invocation via HTTP call

we can copy and notebook you like as reference to newly onboarded Premium users (for sure they can add there own as well)

the API endpoint is the same for all Premium users so you can centrally link a UI for invocation available to all Premium users - but they have to add their JupyterHub API token there so we can internally route to the appropriate endpoint

[juansensio]

Thanks, I will add these instructions in the docs as well.

[earthpulse]

Tried, and worked fine. We can close this issue and open new ones for particular models/datasets we want to support.