Nike-Inc / brickflow

Pythonic Programming Framework to orchestrate jobs in Databricks Workflow

https://engineering.nike.com/brickflow/

Apache License 2.0

183 stars 36 forks source link

[fix] AutosysSensor: Handle the situations where `lastEndUTC` is empty or None #113

Closed maxim-mityutko closed 3 months ago

maxim-mityutko commented 3 months ago

Description

Handle the situations where lastEndUTC data is None plus add tests to test the overall logic and error handling.

Related Issue

112

Motivation and Context

Operator is failing if lastEndUTC is empty.

How Has This Been Tested?

A set of local tests has been implemented + used custom wheel in the workflow

Screenshots (if appropriate):

Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.