Open nachatz opened 3 months ago
This is indeed very handy feature. However, we should enable this feature only for local development.
for higher environments, it does not make sense to deploy only set of workflows and should be part of version control.
Is your feature request related to a problem? Please describe.
Currently, when deploying a given project in Brickflow, it bundles and deploys all workflows under the project. Ideally, for versioned individual workflows we'd like to have the option to deploy just 1 workflow at a time. Ex.
brickflow projects deploy --project Project1 -e local
deploysworkflow_a
andworkflow_b
, a nice addition would be to have the option to only deployworkflow_a
orworkflow_b
.Describe the solution you'd like
brickflow projects deploy --project Project1 --workflow workflow_a -e local
with--workflow
as optionalWhere the bundled path could then possibly include the workflow name
...user/project/workflow/env
Describe alternatives you've considered The alternative is creating a new project per workflow. In our case where we want a ETL pipeline's stages very carefully bundled, versioned, and released this yields 3 new projects at minimum per pipeline (with 3+ pipelines this yields 9+ minimum total projects increasing codebase size & complexity).