New Docker rate limiting affects build success

[Eder-K]

Background

Docker has recently enacted a new policy on how often a given account is allowed to pull images from DockerHub: https://www.docker.com/increase-rate-limits?utm_source=docker&utm_medium=web%20referral&utm_campaign=increase%20rate%20limit&utm_budget=

Effect on testing

Unfortunately for us, that means that builds are more likely to fail due to no fault of their own, see example: https://www.travis-ci.com/github/precice/systemtests/jobs/463952291#L353.

Given the error pops up only after several pull attempts are made, I am not sure whether or not these additional requests count toward the limit. In any case, the number of failures due to this new rate limiter seems suspiciously high at the moment.

Possible Solutions

Here are some pointers as to how we could get around this:

• Make sure we are pulling images while logged into Docker. This raises the rate limit from 100 pulls/6 hours to 200 pulls/6 hours. It appears that we are currently pulling anonymously while building adapters/precice images, for example. Note also this answer from the FAQ:

  Will I be rate-limited if I’m logged in but someone else on my IP hits the anonymous limit? No, users who are logged in and authenticating their pull requests will be limited based on their user account only. If an anonymous user on your IP is limited, you will not be affected as long as your requests are authenticated and you have not hit your own limit.

• Check our current pull consumption and figure out if something wonky is going on.

• Figure out a way to conserve pull requests. The FAQ notes:

  Does it matter what image I am pulling? No, all images are treated equally. The limits are entirely based on the account level of the user doing the pull, not the account level of the repository owner. Despite this, it might be possible to avoid some image pulls, specifically those of the small solver images (we are unlikely to get around pulling full ubuntu/precice-build/adapter-build images).

Additional suggestions are appreciated :+1: .

[Eder-K]

Looking at the build history after merging #271, it seems that pulling as the user precice has resolved the issue (for now). I feel that this can be closed for now, though I expect this error to pop up in the future if tests are added/our docker pull rate increases further.