Possible rate limiting issues of upstream image pulls from dockerhub

[segfault16]

Dockerhub announced they will limit image pulls: https://www.docker.com/blog/scaling-docker-to-serve-millions-more-developers-network-egress/?mkt_tok=eyJpIjoiTW1WbE5qaGlORFJoTXprMSIsInQiOiJ1RllHNk9Tclk4MWRpdzRsMWFvU3RwU05EamtcL05adTFYbzFhTlN6cWE0VUxBd2dCUVd0WTd1ZGU1alBuMCt2ZFdkUjZuWU5GelBEbnhISzZrN3IycVIxTk1VekszR0VHUFl4KzRuVnhDWk1lU0ZVSDRPYXp5bXZMcTRNanNwQXoifQ%3D%3D

Especially in setups with a security proxy in place the rate limit can be hit quite easily for anonymous pulls, since the rate limit is applied by IP then (and in case of a security proxy the same IPs are shared by a large number of people and systems in a company).

[michaelsauter]

Reading the announcement more closely, here are my key takeaways:

the following pull rate limits to Docker subscription plans that will take effect November 1, 2020

There is a small tradeoff – if you pull an image you already have, this is still counted even if you don’t download the layers

So the main issue we'll face will come from application builds which usually depend on community images from Docker Hub like alpine. Every build might add +1 to the count against the rate limit. However, looking at the builds more closely (https://github.com/opendevstack/ods-quickstarters/blob/master/common/ocp-config/component-environment/component-template.yml#L67-L89), there is no force pull - which means that there should not be a pull if the image is present on the node.

Further to builds, every deployment of a public image (e.g. Postgres) which has imagePullPolicy: Always set will also add +1 to the count against the rate limit.

So overall I think this is not a catastrophe, but we should be looking at possible solution. E.g. using base images from other registries such as quay.io and registry.redhat.io (however quay.io might need to be enabled first).

[clemensutschig]

As we build the base images for ods that are used then across the cluster .. we should be save, no?

Michael Sauter notifications@github.com schrieb am Di., 25. Aug. 2020, 09:43:

Reading the announcement more closely, here are my key takeaways:

the following pull rate limits to Docker subscription plans that will take effect November 1, 2020

There is a small tradeoff – if you pull an image you already have, this is still counted even if you don’t download the layers

So the main issue we'll face will come from application builds which usually depend on community images from Docker Hub like alpine. Every build might add +1 to the count against the rate limit. However, looking at the builds more closely ( https://github.com/opendevstack/ods-quickstarters/blob/master/common/ocp-config/component-environment/component-template.yml#L67-L89), there is not force pull - which means that there should not be a pull if the image is present on the node.

So overall I think this is not a catastrophe, but we should be looking at possible solution. E.g. using base images from other registries such as quay.io and registry.redhat.io (however quay.io might need to be enabled first).

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/opendevstack/ods-core/issues/780#issuecomment-679861554, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJV7CSCQUKD6R7E2XRZRSFDSCNTSXANCNFSM4QKKI7EA .

[michaelsauter]

@clemensutschig For agent images yes. For applications the "default" base images are defined in the quickstarters - they are typically from Docker Hub directly. But the cluster nodes should act as caches avoiding pulls as much as possible.

[michaelsauter]

Moved to ods-quickstarters as this relates to the base images of the components. https://github.com/opendevstack/ods-quickstarters/issues/7 is potentially related.