Nike-Inc / brickflow

Pythonic Programming Framework to orchestrate jobs in Databricks Workflow

https://engineering.nike.com/brickflow/

Apache License 2.0

183 stars 36 forks source link

[fix] WorkflowTaskDependecySensor fails if the task execution has not started #123

Closed maxim-mityutko closed 3 months ago

maxim-mityutko commented 3 months ago

Description

Check if task_state is not None, otherwise log and continue poking.

Related Issue

122

Motivation and Context

The sensor will otherwise fail if the task execution has not yet started (e.g. the task is in Pending state)

How Has This Been Tested?

TDD

Screenshots (if appropriate):

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.