search

Nike-Inc / brickflow

Pythonic Programming Framework to orchestrate jobs in Databricks Workflow
https://engineering.nike.com/brickflow/
Apache License 2.0
187 stars 44 forks source link

[FEATURE] Add timeout parameter to workflow class #82

Closed brent-johnson closed 9 months ago

brent-johnson commented 9 months ago

Is your feature request related to a problem? Please describe. Workflow timeout_seconds is implemented in the Bundles Model but is missing from the Engine Workflow (brickflow/engine/workflow.py) this means that Brickflow cannot be used to create workflows with runtime timeouts.

Cloud Information

Describe alternatives you've considered The alternative to this would be to not use brickflow Additional context timeout_seconds: Optional[float] = None

brent-johnson commented 9 months ago

I will be opening a PR with the code change for this issue tomorrow.

brent-johnson commented 9 months ago

https://github.com/Nike-Inc/brickflow/pull/83