Exponential backoff for snapshot recovery

[marvinmarnold]

Notes We are currently using a huge amount of bandwidth on Cloudflare. We believe this is because of our snapshot logic greedily pulling images. We want to better understand what is driving Cloudflare usage and make requests in a more conservative way.

Need to confirm why we are downloading so much.

• Caused by failures or retrying even on success?
• Could this be happening from miners that aren't ours

Acceptance Criteria

• Add custom user agent so we can be sure who is making requests
• Add version to requests so we know which code is healthy

[vpetersson]

We discovered in the grooming that this one is trickier that it seems. Ideally, the retry logic should be done inside of miner code (which is written in Erlang unfortunately).

[shawaj]

@marvinmarnold @vpetersson we can probably close this now the issues with cloudflare are sorted? dunno what you think?

[vpetersson]

Yeah agreed. Not super relevant anymore.