Python CDK `should_retry` behavior nullified by `default_backoff_handler`

[wjwatkinson]

Environment

• Airbyte version: example is 0.35.58-alpha
• OS Version / Instance: macOS, GCP n2.
• Deployment: Docker
• Source Connector and version: Kyriba - https://github.com/wjwatkinson/airbyte/blob/source-kyriba/airbyte-integrations/connectors/source-kyriba/source_kyriba/source.py
• Destination Connector and version: BigQuery
• Severity: Medium
• Step where error happened: Sync job

  Current Behavior

  The source should_retry method evaluates to True, but the request is never retried because should_give_up in rate_limiting.py evaluates to True.

Expected Behavior

If should_retry evaluates to True the request should be retried.

Logs

logs-50.txt

2022-03-19 21:52:58 source > Giving up for returned HTTP status: 401
2022-03-19 21:52:58 source > Giving up _send(...) after 1 tries (airbyte_cdk.sources.streams.http.exceptions.DefaultBackoffException)

Steps to Reproduce

1. Overwrite the should_retry method in your HttpStream to include a status between 400 and 500 other than 429.
2. Run your stream, so it encounters this error status.
3. Watch the stream not retry.

Are you willing to submit a PR?

Yes

[philippeboyd]

any update on this alafanechere?