Nike-Inc / brickflow

Pythonic Programming Framework to orchestrate jobs in Databricks Workflow
https://engineering.nike.com/brickflow/
Apache License 2.0
183 stars 39 forks source link

[BUG] Faulty `execution_date` parser in AutosysSensor #88

Closed maxim-mityutko closed 7 months ago

maxim-mityutko commented 7 months ago

Describe the bug The below code in AutosysSensor tries to parse the execution timestamp based on a specific format. https://github.com/Nike-Inc/brickflow/blob/548b6cb47330f3e59c08839f91b9282750332ca4/brickflow_plugins/airflow/operators/external_tasks.py#L401 However we noticed that this format is different depending if task is executed on a manual workflow run or based on schedule.

To Reproduce Steps to reproduce the behavior: Add and setup AutosysSensor in the workflow, run the workflow manually.

Expected behavior The execution date should be parsed correctly irrespective of the string format.

Screenshots If applicable, add screenshots to help explain your problem.

Cloud Information

Desktop (please complete the following information):

Additional context Add any other context about the problem here.