iterative / gto

🏷️ Git Tag Ops. Turn your Git repository into Artifact Registry or Model Registry.
https://dvc.org/doc/gto
Apache License 2.0
142 stars 16 forks source link

Feature Request: Deployment Stage DAG Enforcement #321

Open abpwrs opened 1 year ago

abpwrs commented 1 year ago

I'm interested in enforcing that model versions are assigned to "lower" deployment environments (closer to dev) before they are allowed to be assigned to "higher" environments (closer to prod). I've drafted out an example below, and was curious if this feature would align w/ the goals for GTO?

graph LR;
    dev-->qa;
    qa-->stage;
    stage-->prod;
    hot-stage-->prod;

For instance, to enforce the promotion DAG above, maybe an extension of the existing STAGES field in the .gto file:

STAGES:
  - dev
  - qa:
      promotes_from:
        - dev
  - stage:
      promotes_from:
        - qa
  - hot-stage
  - prod:
      promotes_from:
        - stage
        - hot-stage

Then during normal use you'd see:

gto register hello-world
gto assign hello-world --version v0.0.1 --stage dev
gto assign hello-world --version v0.0.1 --stage qa

# some commits and dev work
gto register hello-world --bump-patch

# this should fail, as it should enforce assignment to `dev` before assigning to `qa`
gto assign hello-world --version v0.0.2 --stage qa

# ?? maybe allow `--force` to override DAG rules ??
gto assign hello-world --version v0.0.2 --stage qa --force
aguschin commented 1 year ago

Hi! Looks like an interesting use case indeed. I think we can implement this in GTO.

Do you use GTO already and need this to support your workflow? Or just considering whether to try it out?

Also, want to clarify one thing: after running

gto register hello-world
gto assign hello-world --version v0.0.1 --stage dev
gto assign hello-world --version v0.0.1 --stage qa

you're expecting v0.0.1 to be in both dev and qa, right?

abpwrs commented 1 year ago

Do you use GTO already and need this to support your workflow? Or just considering whether to try it out?

Hi @aguschin! This isn't part of a workflow yet. I'm just exploring data/model registry solutions, and, in learning how GTO works, I found that I'd want some type of deployment stage protection.

Also, want to clarify one thing: after running

gto register hello-world
gto assign hello-world --version v0.0.1 --stage dev
gto assign hello-world --version v0.0.1 --stage qa

you're expecting v0.0.1 to be in both dev and qa, right?

Correct, I'd expect v0.0.1 to be in dev and qa :+1:

aguschin commented 1 year ago

Sounds good! Let me know if you'll need this - I can implement it shortly if you decide to use GTO. Adding to the backlog for now.