Sometimes, API requests fail and the code will retry. However, the retry attempt sometimes produces unexpected results. For example, during the item fetching stage, if a timeout error occurs, the code will retry but instead, the item fetching process stops as if there were no more items to fetch.
This issue may be fixed by implementing more robust retry code. Instead of the existing solution, use the built-in retry functionality of the requests library. This was already implemented for file download retries in #83. This issue is to implement the same for all uses of the requests library.
Is there an existing issue for this?
Description of the bug
Sometimes, API requests fail and the code will retry. However, the retry attempt sometimes produces unexpected results. For example, during the item fetching stage, if a timeout error occurs, the code will retry but instead, the item fetching process stops as if there were no more items to fetch.
This issue may be fixed by implementing more robust retry code. Instead of the existing solution, use the built-in retry functionality of the requests library. This was already implemented for file download retries in #83. This issue is to implement the same for all uses of the requests library.
Steps To Reproduce
See above.