allegroai / clearml

ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution
https://clear.ml/docs
Apache License 2.0
5.61k stars 651 forks source link

Updating pipeline without re-running task #1121

Open LifeBoey opened 1 year ago

LifeBoey commented 1 year ago

Hi there,

I'd like to ask here the same question I did in slack (https://clearml.slack.com/archives/C028LNAAR5H/p1695283353097409 ), which is regarding updating the pipeline and running the updated pipeline, without having to re-run the affected tasks.

If I were to change the code in the pipeline, I find that simply running the pipeline again would use the code from the previously ran instance, even after updating the code in the git repository. In order for the pipeline to acknowledge the new code, it'd require me to run the individual task(s) that were affected by it.

I was wondering if there is just a faster way to update the pipeline code without having to go through the tedium of running the tasks again. Much appreciated!

eugen-ajechiloae-clearml commented 1 year ago

Hi @LifeBoey ! Just for the record, you can override repo info in add_step using task_overrides https://clear.ml/docs/latest/docs/references/sdk/automation_controller_pipelinecontroller#add_step