astronomer / airflow-provider-great-expectations

Great Expectations Airflow operator
http://greatexpectations.io
Apache License 2.0
159 stars 54 forks source link

Cannot run validation on BigQuery #119

Open 1cadumagalhaes opened 11 months ago

1cadumagalhaes commented 11 months ago

Hello!

I'm trying to execute Great Expectations in my airflow instance to run some validations in my BigQuery tables. There's not much content about this on the internet, and I even found some references to an old GreatExpectationsBigQueryOperator.

I've setup my connection on Airfow as a Google Cloud connection and it works for other things (such as querying data), but on the GreatExpectationsOperator I get this error below:

ERROR - Failed to execute job 118 for task gx_validate_products (Conn type: google_cloud_platform is not supported.; 425)

Here's my code:

 gx_validate_products = GreatExpectationsOperator(
        task_id="gx_validate_products",
        conn_id="gcp_connection",
        data_context_root_dir="great_expectations",
        data_asset_name="sample_ecommerce.products",
        expectation_suite_name="raw_products",
        return_json_dict=True,
    )
plotti commented 9 months ago

I am having the same issue, is there some feedback available on this?

tamarasommerhalder commented 9 months ago

hello, I am having the same issue. Looking forward for feedback on this. Thanks.