Closed jlaneve closed 1 year ago
2.6.1
The type of the libraries field in the DatabricksSubmitRunOperator is incorrect. According to the Databricks docs, the values should look more like:
[ {"pypi": {"package": "simplejson"}}, {"pypi": {"package": "Faker"}}, ]
as opposed to what the type hint implies:
[ {"pypi": "simplejson"}, {"pypi": "Faker"}, ]
https://github.com/apache/airflow/blob/providers-databricks/4.2.0/airflow/providers/databricks/operators/databricks.py#L306
No response
n/a
macOS
Astronomer
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
cc: @alexott
I will check this
Apache Airflow version
2.6.1
What happened
The type of the libraries field in the DatabricksSubmitRunOperator is incorrect. According to the Databricks docs, the values should look more like:
as opposed to what the type hint implies:
https://github.com/apache/airflow/blob/providers-databricks/4.2.0/airflow/providers/databricks/operators/databricks.py#L306
What you think should happen instead
No response
How to reproduce
n/a
Operating System
macOS
Versions of Apache Airflow Providers
No response
Deployment
Astronomer
Deployment details
No response
Anything else
No response
Are you willing to submit PR?
Code of Conduct