apify / keboola-ex-apify

Apify extractor for Keboola Connection
https://components.keboola.com/components/apify.apify
Apache License 2.0
0 stars 0 forks source link

Use exponential backoff in case API requests failed #47

Open drobnikj opened 6 months ago

drobnikj commented 6 months ago

The problem is that when downloading one part of the dataset failed, the whole import into Keboola failed. This should be fixed using exponential backoff retries. The exponential back off is included in apify-client, so just use the client correctly should fix this.

see https://apify.slack.com/archives/CGZSN9DQC/p1713945183893599