airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.04k stars 4.11k forks source link

connectors-ci: make retries not send sentry errors #29321

Open alafanechere opened 1 year ago

alafanechere commented 1 year ago

We implemented a retry mechanism for transient dagger error. It has proven to be working: e.g; this error :

failed to authorize: failed to fetch oauth token: unexpected status from POST request to https://auth.docker.io/token: 504 Gateway Timeout

was retried in the pipeline and the execution ended up being successful.

Still, a sentry error was created, probably because of a not optimal logging strategy that logs with "error" level the retries.

Solution

Make sure retries log using the WARN level. Make sure final failures log using the ERROR level

Questions

@pedroslopez what makes Sentry capture an error in the logs: the fact it has an exc_info attribute and/or its log level. In other words: will a WARN log message with exc_info lead to a sentry event?

sentry-io[bot] commented 1 year ago

Sentry issue: CONNECTORS-CI-3J

pedroslopez commented 1 year ago

@pedroslopez what makes Sentry capture an error in the logs: the fact it has an exc_info attribute and/or its log level. In other words: will a WARN log message with exc_info lead to a sentry event?

@alafanechere Only error logs with exc_info lead to a sentry error