Nike-Inc / brickflow

Pythonic Programming Framework to orchestrate jobs in Databricks Workflow
https://engineering.nike.com/brickflow/
Apache License 2.0
183 stars 36 forks source link

[FEATURE] If/Else Task #135

Closed RajuGujjalapati closed 1 month ago

RajuGujjalapati commented 2 months ago

Is your feature request related to a problem? Please describe. Databricks now supports creating a "If/else" task, to run a part of a job DAG based on a boolean expression. The expression consists of a boolean operator and a pair of operands, where the operands might reference job or task state

Cloud Information

Describe the solution you'd like providing If/else option to create workflows, also will add a new if_else_condition_task type. It would be nice to have something like this:

Describe alternatives you've considered I've looked at invoking dependent jobs with the Databricks API

Additional context

image