astronomer / astro-provider-databricks

Orchestrate your Databricks notebooks in Airflow and execute them as Databricks Workflows
Apache License 2.0
21 stars 11 forks source link

Support Airflow version 2.2.4, 2.3 and 2.4 and enable CI run on it #3

Closed pankajkoti closed 1 year ago

pankajkoti commented 1 year ago

At the moment, our customer who requested the Databricks support uses:

Python 3.9
Airflow 2.2.4

(based on Sigma)

However, we are only testing Python 3.8 and Airflow 2.5. We should extend the current CI setup https://github.com/astronomer/astronomer-cosmos/issues/161 (PR https://github.com/astronomer/astronomer-cosmos/pull/167) to also validate against this version of Python and Airflow.

Add needed support in the implementation for supporting Airflow 2.2.4, 2.3 and 2.4 together with Python version 3.8 & 3.9 and enable CI to run on such a matrix.