Open Beat-Nick opened 1 month ago
hi @Beat-Nick , thanks for reporting this issue. We're in the process of contributing the functionalities of this provider to the official Apache Airflow Databricks provider. With the upcoming 6.8.0 release of the Airflow Databricks provider (should hopefully be out in a week's time if the voting passed for the release candidate), all the functionality along with few bug fixes will be available in that provider. This bug you're mentioning has also been fixed. Migrating to the Airflow provider would be simple, just would need to change the import path to the Airflow Databricks provider. We appreciate your patience until then. Please let us know if you'd have further questions here or need more details.
@pankajkoti, its a great initiative to move this under Databricks provider. I am wondering if there is any plan/roadmap to address tech debts of this plugin that you can share which help us to contribute to improve
hi @VenkatH , the contribution to the Airflow Databricks provider is complete and all the existing functionalities + bug fixes have been released with the 6.8.0 version.
You might want to try that release and see if all issues you're observing are fixed already. If not, please feel free to report issues and we could work on fixing those together.
Happy to answer if you'd have more questions!
We've just switched over to this repo and while everything works when I click on either of the task failure repair buttons I get a blank page showing this error:
Cannot GET /repair_databricks_job
Taking a look it seems the plugin returns the URL as designed but it doesn't have an actual destination.
See Databricks Job Run
works as intended so I know our connection is setup correctly.We're running Astro 9.5.0 and the DAG and tasks all run successfully outside this one issue.