astronomer / airflow-provider-great-expectations

Great Expectations Airflow operator
http://greatexpectations.io
Apache License 2.0
157 stars 53 forks source link

Fix make_connection_configuration for MSSQL (adding a driver) #113

Closed TJaniF closed 11 months ago

TJaniF commented 1 year ago

Hi :)

This came up in the GX Slack twice. Using the GXO with a MSSQL conn_id currently does not work because there is no driver being passed via the URI. If fails with:

sqlalchemy.exc.InterfaceError: (pyodbc.InterfaceError) ('IM002', '[IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified (0) (SQLDriverConnect)')

There was also an issue with the connection needing a database argument separate from the schema passed through the operator.

This PR should fix that. I added the possibility to pass a driver via the connection extras, if none is passed ODBC Driver 17 for SQL Server is passed, which is what worked for me when testing with the mcr.microsoft.com/mssql/server:2022-latest image. I tested this change with Postgres and MSSQL. :)

@phanikumv I hope it is ok to tag you for review on this, Viraj told me you might be able to help :)

TJaniF commented 1 year ago

@Lee-W apologies, I tested the operator again and found out that the connection only accepts the driver when a database is specified in the URI. Added the MSSQL default database master in case a user has none specified in their connection (which is possible because the connection works with the MsSqlOperator without naming the database). No other changes 😅

TJaniF commented 11 months ago

@phanikumv thank you! 😊