Control renku run behaviour from inside python code

SwissDataScienceCenter / renku-python

A Python library for the Renku collaborative data science platform.

https://renku-python.readthedocs.io/

Apache License 2.0

37 stars 29 forks source link

Open Panaetius opened 4 years ago

Panaetius commented 4 years ago

It's be great if users could influence the behavior of renku run script.py ... from inside script.py

This could include:

Not creating a metadata/KG entry for the run (e.g. if results were too bad or there was an exception)
Splitting a run into several stages
Associating a transaction id with a run and only committing metadata changes once the transaction is finished (might happen in a separate run)

olevski commented 2 years ago

This can be done in several ways:

Add to the metadata store in the repo to indicate that a run is underway but that the commit should happen later or not at all
Add something to the files renku creates to handle parameters and inputs