search

carvel-dev / kbld

kbld seamlessly incorporates image building and image pushing into your development and deployment workflows
https://carvel.dev/kbld
Apache License 2.0
294 stars 41 forks source link

Docker API Rate Limit #456

Closed antwacky closed 9 months ago

antwacky commented 9 months ago

Does anyone know if the KBLD queries count towards the Docker API rate limit? It does seem to.

Which brings me to my next question, does KBLD have a way to use a registry mirror? We have registry mirrors set up in our Docker daemon config to avoid hitting rate limits when pulling images and in order to use KBLD - we would need a way to make KBLD use/respect configured registry mirrors.

antwacky commented 9 months ago

Reading the Dockerhub API rate limiting doc, it seems that two GET requests count as a pull.

Does KBLD use a HEAD instead which is not rate limited?

joaopapereira commented 9 months ago

By default, kbld uses GET, not HEAD. In the current implementation, there is no way to automatically have a mirror registry for DockerHub. You could accomplish that at the OS level by setting the host file or something similar.

kbld could eventually be enhanced to support some Environment variable that would do this for you.

antwacky commented 9 months ago

I've raised a pull request to change the method from GET to HEAD.

I've tested and can see the DockerHub ratelimit-remaining being reduced after using the current version, and unaffected using the pull request fork.

antwacky commented 9 months ago

@joaopapereira do you think someone could review my pull request? We will struggle to adopt kbld with the current implementation draining the dockerhub rate limit.

joaopapereira commented 9 months ago

Sorry for the delay but I was out for a couple of days, I left a review in the PR