simple retry for fetch

[bkmgit]

Perhaps can update the code in helpers.py, to use aiohttp and asyncio

[mku11]

I can help with this, though I'm not sure how it would work out with an asyncio API. Wouldn't async propagate to the caller?

Otherwise, do you want a user param with a default 1,2, or more? or a decorator if you have other net request functions that it can be applied to?

How about exponential backoff or is it getting unnecessarily complicated and moving away from being tiny? I'm assuming most of the links would be in cloud storage/APIs. If you feel net congestion is a probable case I can add a delay.

[geohot]

@Qazalin this is not a good first issue. On the surface it sounds like one because it's simple, but read the replies and see why it isn't.

A good first issue is something where there's pretty much one way to do it and it's easy for a person to tell themselves if they did it correctly.

Compare it to an issue like #7176

[bkmgit]

An example implementation is at https://github.com/Thinklab-SJTU/pygmtools/pull/73/files based on

• https://likegeeks.com/downloading-files-using-python/
• https://docs.aiohttp.org/en/latest/http_request_lifecycle.html
• https://codereview.stackexchange.com/questions/259112/async-download-of-files
• https://docs.aiohttp.org/en/stable/client_quickstart.html

[mku11]

I have a solution ready for this with the following capabilities, let me know if it suits you. It helps when these questions are answered before submitting useless PRs.

1. param 'retries' with default value = 1 and internal constraint of max 4 retries
2. retry based on http code list: 408, 425, 429, 500, 502, 503, 504.
3. simple expo backoff with a multiplier of 2
4. No asyncio.
5. No decorators