search

astronomer / astro-provider-databricks

Orchestrate your Databricks notebooks in Airflow and execute them as Databricks Workflows
Apache License 2.0
21 stars 11 forks source link

Repair Failed Tasks & Repair Task button fails with bad URL #83

Open Beat-Nick opened 1 month ago

Beat-Nick commented 1 month ago

We've just switched over to this repo and while everything works when I click on either of the task failure repair buttons I get a blank page showing this error: Cannot GET /repair_databricks_job

Taking a look it seems the plugin returns the URL as designed but it doesn't have an actual destination.

See Databricks Job Run works as intended so I know our connection is setup correctly.

We're running Astro 9.5.0 and the DAG and tasks all run successfully outside this one issue.

pankajkoti commented 1 month ago

hi @Beat-Nick , thanks for reporting this issue. We're in the process of contributing the functionalities of this provider to the official Apache Airflow Databricks provider. With the upcoming 6.8.0 release of the Airflow Databricks provider (should hopefully be out in a week's time if the voting passed for the release candidate), all the functionality along with few bug fixes will be available in that provider. This bug you're mentioning has also been fixed. Migrating to the Airflow provider would be simple, just would need to change the import path to the Airflow Databricks provider. We appreciate your patience until then. Please let us know if you'd have further questions here or need more details.

VenkatH commented 1 month ago

@pankajkoti, its a great initiative to move this under Databricks provider. I am wondering if there is any plan/roadmap to address tech debts of this plugin that you can share which help us to contribute to improve

pankajkoti commented 1 month ago

hi @VenkatH , the contribution to the Airflow Databricks provider is complete and all the existing functionalities + bug fixes have been released with the 6.8.0 version.

You might want to try that release and see if all issues you're observing are fixed already. If not, please feel free to report issues and we could work on fixing those together.

Happy to answer if you'd have more questions!