search

sodadata / soda-sql

Soda SQL and Soda Spark have been deprecated and replaced by Soda Core. docs.soda.io/soda-core/overview.html
https://docs.soda.io/
Apache License 2.0
60 stars 15 forks source link

Marking tasks with error as successful on airflow when using PythonVirtualEnvOperator #1

Open pedro-gil opened 2 years ago

pedro-gil commented 2 years ago

I've been running soda-sql-bigquery version 2.1.5 using the PythonVirtualEnvOperator on an Airflow environment.

There was a change in the service account key, so the credentials did fail presenting the following error: "[ScanError(message='Exception during scan', exception=RefreshError('invalid_grant: Invalid JWT Signature.', {'error': 'invalid_grant', 'error_description': 'Invalid JWT Signature.'}))]"

It also logged the scan with problems with this message: "There were errors in the scan execution."

However, when it comes to the airflow dag, this task was marked as successful. It would be great to know when the scans fail without having to check the logs.

HansBambel commented 2 years ago

I have the same issue that the errors in a soda-test does not error out and appear as if all tests passed.

I am using soda with a BashOperator