astronomer / astro-provider-databricks

Orchestrate your Databricks notebooks in Airflow and execute them as Databricks Workflows
Apache License 2.0
20 stars 10 forks source link

Fix repairing tasks declared in inner task groups #49

Closed tatiana closed 1 year ago

tatiana commented 1 year ago

Fix two problems with the repair feature, related to the following buttons:

A customer reported these problems, and this behaviour happened with all Airflow versions (from 2.2.4 until 2.6)

This change was validated through unit tests and also manually, using Ubuntu 22.x, by following these steps:

The fix for the second issue can be visually seen in this screenshot: Screenshot from 2023-06-16 15-26-40

DIsclaimer: This change was also tested on Airflow 2.2.4, and observed that when the feature repair is used in this version, the Airflow tasks are not cleared - but the Databricks job is repaired.

codecov-commenter commented 1 year ago

Codecov Report

Patch coverage: 70.00% and project coverage change: -0.43 :warning:

Comparison is base (d6c9c22) 78.33% compared to head (8e22e6b) 77.91%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #49 +/- ## ========================================== - Coverage 78.33% 77.91% -0.43% ========================================== Files 5 5 Lines 480 498 +18 Branches 64 66 +2 ========================================== + Hits 376 388 +12 - Misses 83 88 +5 - Partials 21 22 +1 ``` | [Impacted Files](https://app.codecov.io/gh/astronomer/astro-provider-databricks/pull/49?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=astronomer) | Coverage Δ | | |---|---|---| | [src/astro\_databricks/plugins/plugin.py](https://app.codecov.io/gh/astronomer/astro-provider-databricks/pull/49?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=astronomer#diff-c3JjL2FzdHJvX2RhdGFicmlja3MvcGx1Z2lucy9wbHVnaW4ucHk=) | `68.46% <70.00%> (-0.16%)` | :arrow_down: |

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.