build/common: Add remote_fetch curl wrapper

[morucci]

This patch purpose remote_fetch in order to make curl/wget usage consistent between calls. The main point is to use the curl retry mechanism to bypass temporary remote failures.

This can be a start point to make our build process resilient to upstream failure ... Let us know your thoughts about that. We start to use it to build SF images.

[ErwanAliasr1]

On what calls do you aim to use it ? all of them ?

[morucci]

Basically to replace each direct call to wget or curl (at least in *.install files). This is to solve those issues:

• avoid build fail if an upstream point return like a 5\ error, by using the retry feature of curl. This aims to stabilise image building.
• Avoid unconsistency between component fetching method (eg. sometime wget, sometime curl, sometime option X, sometime option XY ...)

Have a look here http://softwarefactory.enovance.com/_r/%7C/c/428/

[morucci]

For -m, --max-time I'm not sure about this one as it seems to be related to the maximum amount of time to perform an operation. So I wont be able to tell you what to specify as . A fetch can take long if the network is slow but if the fetch is working the this not really a problem. So for me the "-m" option does not solve the problem.

For the retry-max-time option why not. If we use something like 120 to this option we won't loop trying to fetch the resource after 120 s. This is the same as I purpose expect that with --retry-max-time we use the "exponential backoff algorithm" instead of fix 10 second 12 times.

[ErwanAliasr1]

I like the idea, please make a PR with the full change.

[fredericlepied]

@morucci still working on this ?

[ErwanAliasr1]

@morucci hey, still interested in this PR ?