search

astronomer / airflow-provider-great-expectations

Great Expectations Airflow operator
http://greatexpectations.io
Apache License 2.0
157 stars 53 forks source link

GreatExpectationsOperator is overriding database name with schema #107

Open qwertyyuiop01 opened 1 year ago

qwertyyuiop01 commented 1 year ago

hello,

Usecase: Database name is different from schema names and multiple schemas are available..

  1. Trying to connect to redshift db name: main (using conn_id and database name is passed in airflow UI) is failing when schema= "login" or data_asset_name="schema.table_name" is passed as parameter to GreatExpectationsOperator

Error: connection failed for database login.

  1. If schema is not passed as parameter or with data_asset_name then seeing

InvalidSchemaName: schema "main" does not exist

Screenshot 2023-04-14 at 2 22 21 PM
mpgreg commented 1 year ago

Same thing happens with postgres.

mpgreg commented 1 year ago

Need to update regarding the change to postgres hook. https://github.com/apache/airflow/pull/26744. Schema should be passed as option in the uri. Such as:

AIRFLOW_CONN_POSTGRES_DEFAULT='postgresql://postgres:postgres@host:5432/postgres?options=-csearch_path%3Dpublic'