search

MeltanoLabs / tap-intacct

1 stars 0 forks source link

fix: retry on intermittent `Bad request for path: /ia/xml/xmlgw.phtml` #9

Open pnadolny13 opened 5 days ago

pnadolny13 commented 5 days ago

Intacct seems to periodically throw this error which currently get raised and ends the sync singer_sdk.exceptions.FatalAPIError: 400 Client Error: Bad request for path: /ia/xml/xmlgw.phtml. We should use backoff handling to wait and retry. Its a bit tricky because 400's can be a legit bad request or this intermittent bad request so we should find a way to differentiate if we can otherwise its probably best to start with limited retry logic to avoid multiple retries on a legitimate bad request.

cc @edgarrmondragon

edgarrmondragon commented 4 days ago

we should find a way to differentiate if we can we definitely should try that. Maybe a header or something in response body.

Otherwise like you say we could try updating the extra_retry_statuses 

from http import HTTPStatus

class IntacctStream(RESTStream):
    extra_retry_statuses = [
        HTTPStatus.BAD_REQUEST,
        HTTPStatus.TOO_MANY_REQUESTS,
    ]

which should the request be retried up to 5 times in case there's some server-side flakiness.