numaproj / numaplane

Control Plane for Numaproj
Apache License 2.0
13 stars 2 forks source link

Pipeline-level USDE #385

Open juliev0 opened 2 weeks ago

juliev0 commented 2 weeks ago

Summary

It sounds like Vigith and others prefer a pipeline-level USDE. We still need to resolve how it works to update Numaflow Controller. For updating isbsvc, they want to focus on the Intuit use case, which is that there's a 1:1 mapping between Pipeline and isbsvc (but still need to figure out how it works for open source community).


Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

juliev0 commented 2 weeks ago

One option is to say that we always upgrade the Numaflow Controller and the isbsvc the "most conservative" way designated for any Pipelines, where "PPND" is more conservative than "no strategy", and "Progressive" is more conservative than "PPND".

In this case, I think we'd need to look at having some namespace-level "In-Progress-Strategy" state to prevent for example a Numaflow Controller Rollout updating with "Progressive" while a Pipeline is updating with "PPND", but it's complicated because 2 Pipelines theoretically could upgrade at the same time with different strategies.