Open sanderegg opened 1 month ago
Docker log in per request
vs Private registry as pull through cache to Dockerhub
Docker log in per request - need to be done per request (every time) - need to log in on every machine
Private registry as pull through cache to Dockerhub + once set up works automatically out-of-the-box with no overhead - OPS need to guarantee private registry uptime - OPS need to properly loadbalance and scale private registry - OPS need to maintain disk space of private registries - once registry is down no image can be pulled via regsitry - daemon.json needs to be configured on every machine
Decision
Based on this comparison, I decide to go on with Docker log in per request
approach. Even though I would rather have something working automatically out-of-the-box but the cost is too high
@mrnicegyu11, @sanderegg, let me know what are your thoughts on this matter
Does it matter which user "runs" docker container [linux] process? (unrelated to docker swarm)
--with-registry-auth
option, the pulling of images is authenticatedDo I need to issue docker login
on every docker swarm node?
docker service create --with-registry-auth ...
Do I need to issue docker login every time I ssh into machine?
~/.docker/config.json
file apparentlyDo I need to issue docker login on the machine or docker swarm already have login information (in case it was run before with --with-registry-auth)?
docker login
+ --with-registry-auth
option)docker service ...
and docker stack ...
commands with --with-registry-auth
optionSwarm Cluster:
Case 1
Setup:
docker service create --mode global --name hello-world hello-world
is executed on managerOutcome:
Case 2
Setup:
docker service create --mode global --name hello-world hello-world
is executed on managerOutcome:
Case 3
Setup:
docker service create --mode global --name hello-world hello-world
is executed on managerOutcome:
Case 4
Setup:
docker service create --with-registry-auth --mode global --name hello-world hello-world
is executed on managerOutcome:
Case 4
Setup:
docker service create --mode global --name hello-world hello-world
is executed on managerOutcome:
Case 5. All machines are logged in. No --with-registry-auth
option
Setup:
docker service create --mode global --name hello-world hello-world
is executed on managerOutcome:
Case 6. All machines are [docker] logged out. With --with-registry-auth
option. The image is present on all machines
Setup:
docker service create --with-registry-auth --mode global --name hello-world hello-world
is executed on managerOutcome:
Docker log in per request
vsPrivate registry as pull through cache to Dockerhub
Docker log in per request - need to be done per request (every time) - need to log in on every machine
Private registry as pull through cache to Dockerhub + once set up works automatically out-of-the-box with no overhead - OPS need to guarantee private registry uptime - OPS need to properly loadbalance and scale private registry - OPS need to maintain disk space of private registries - once registry is down no image can be pulled via regsitry - daemon.json needs to be configured on every machine
Decision Based on this comparison, I decide to go on with
Docker log in per request
approach. Even though I would rather have something working automatically out-of-the-box but the cost is too high@mrnicegyu11, @sanderegg, let me know what are your thoughts on this matter
I would like to discuss these +/-. If the private registry is down that means anyway:
Docker log in per request
vsPrivate registry as pull through cache to Dockerhub
Docker log in per request - need to be done per request (every time) - need to log in on every machine Private registry as pull through cache to Dockerhub + once set up works automatically out-of-the-box with no overhead - OPS need to guarantee private registry uptime - OPS need to properly loadbalance and scale private registry - OPS need to maintain disk space of private registries - once registry is down no image can be pulled via regsitry - daemon.json needs to be configured on every machine Decision Based on this comparison, I decide to go on withDocker log in per request
approach. Even though I would rather have something working automatically out-of-the-box but the cost is too high @mrnicegyu11, @sanderegg, let me know what are your thoughts on this matterI would like to discuss these +/-. If the private registry is down that means anyway:
* nothing can run as all the services are only available there (s4l, isolve, etc...) * OPS side, if the registry is down, then oSparc is anyway able to run nothing, so not sure this changes much, * Costs side, it is probably an improvement, especially if the registry is setup with S3 VPC, and scaled on the regions we are using, * number of pulls toward Dockerhub is down to a minimum
@mrnicegyu11, @sanderegg and I had a discussion and decided to try pull through registry approach. I will do a POC to check how / if it actually works
Useful links:
Insights:
Conclusions
docker image push
docker login
and --with-registry-auth
option to be executedI am in favor of docker login
+ --with-registry-auth
approach at this point of time. Once all our machines are private, we can consider it one more time.
UPD:
Context:
Outcomes:
mirror
registry container is not running, docker pull as if there was no registrymirror
registry without restarting docker. The mirror
registry will be still usedmirror
registry are incorrect, docker will pull as if there was no registrymirror
registry is down, docker will still pull imagesBlocked for AWS Deployments until OPS nodes are private https://github.com/ITISFoundation/osparc-ops-environments/issues/574
Since AWS-master and Tip.science are using private EC2 machines the following issue is arising:
Current possible options:
some references: