search

jupyterhub / mybinder.org-deploy

Deployment config files for mybinder.org
https://mybinder-sre.readthedocs.io/en/latest/index.html
BSD 3-Clause "New" or "Revised" License
76 stars 74 forks source link

Use credentials when pulling from dockerhub #1663

Open betatim opened 4 years ago

betatim commented 4 years ago

Dockerhub will start rate limiting pulls from its registry. The rate limit will be applied based on the IP of the docker client. So all pulls from one cluster will be under the same "account".

There are a few scenarios in which this will touch mybinder.org.

  1. repos that use a custom Dockerfile with a base image that we don't already have
  2. repos that use the github action to pre-build images that we then pull (once) on launch

In addition some of the clusters use dockerhub as their internal registry. This means every launch (with the image not already present on the node) will need to pull, contributing to the rate limit.

These are a few places I think we should go and investigate:

  1. configure k8s so that it uses credentials when pulling from docker hub
  2. configure repo2docker to use credentials when pulling from docker hub
betatim commented 4 years ago

Triggered by https://discourse.jupyter.org/t/dockerhub-pull-limits-mybinder-org/6381/4

minrk commented 4 years ago

Might also be useful to look at image pull secrets for a service account which should have the effect of a 'default' imagePullSecret without having to specify it for every pod separately.

arnim commented 4 years ago

@MridulS we now have the Pro account and use the corresponding credentials for pulling at GESIS - is this correct?

MridulS commented 4 years ago

Yes, and as we don't use DIND to build/pull, we just use authenticated docker ($ docker login) on our host server . cc @bitnik

betatim commented 3 years ago

Looks like the OVH cluster is out of action because it is hitting the docker hub rate limit. I will take a look at adding a image pull secret by hand and if that works figure out how/where to add it in the BinderHub chart.

Right now thinking it should be separate from the "docker registry" credentials we have because you might have a private registry and also want to configure docker hub credentials for pulling (base) images from there. Something I don't know yet is how to ask helm to patch the default service account. Or if it is even sensible to try and do that.