Closed excurs0r closed 6 years ago
Got mail from Telekom. It's not possible. There is a "Cloud Container Engine" with documentation explaining how to install certificates, tag docker images, login with docker and push them to the registry. But at the moment this part is broken.
I believe, there is a misunderstanding, because I am pretty sure that I am the one who wrote you the mail, so I can clear this up. The initial question was: "How to deploay docker containers with terraform to Open Telekom Cloud?" In the mail was the first question from my side: Which environment is used to deploy docker containers? Terraform is only a way to create infrastructure ressources where a docker host or cluster can resides on. There is no way to create docker containers "directly" with Terraform. The mentioned Cloud Container Engine (CCE) is one way how we provide a Platform as a Service (PaaS) for our customers to easily create a Kubernetes based docker cluster and run containers on. But there is no Terraform-interface to create such a CCE-cluster.
Nice to hear from you :) I checked it and you're right, it was your mail. I'd really like to get things started, but I really don't know what I do wrong.
To get things clear for everyone I'll try my best to describe, what I've done and where I failed.
First of all, I'm working on Debian/Buster. I've installed lots of stuff. Amongst other things docker and terraform.
So this is how I understood the deployment process of the infrastructure:
So what did I exactly do:
I started by logging in at https://auth.otc.t-systems.com/authui/login.action#/login. Then I took a look at the Container Registry. There is a button that says "Upload Container Image". I clicked on that and there were three links to documentations. I followed those. At first I created an auth-token, configured docker with "insecure-registries" in (/etc/docker/daemon.json) and tried to login with docker. The command I used for that was:
docker login -u _auth_token -p {generated-token} -e aa {fancy-ip}:{some-port}
This worked sometimes and sometimes not. I was unable to find out why. Restarted docker several times (service and socket). It was like 1 out of 15. There were no typos, because I always reused the old command via bck-i-search (Ctrl+R).
Then I took the next steps.
I added the certificates to have trusted content. I put them in
/usr/share/ca-certificates
Then I updated certificates: sudo update-ca-certificates
Output says they were added.
I also set the environment variables:
DOCKER_CONTENT_TRUST=1
DOCKER_CONTENT_TRUST_SERVER={cool-ip}:{fancy-port}
The next step is to tag a docker image and then push it to container registry.
So I tagged an image. With "docker images" command I checked it again:
{ip}:{port}/something/someimage latest
And now comes the part, where I failed.
docker push {ip}:{port}/something/someimage
And this is the output:
The push refers to a repository [{ip}:{port}/something/someimage]
068f00f601b4: Preparing
ffb1344a0bfb: Preparing
unauthorized: authentication required
The login didn't expire. And the docker error message is not very helpful, because unauthorized means "I know you, but you are not allowed to do that" and authentication required means "I don't know you, please login".
So I started debugging that. First I took a look at /var/log/daemon.log
.
There it says:
Feb 21 13:59:18 machine dockerd[19533]: time="2018-02-21T13:59:18.079352446+01:00" level=error msg="Upload failed: unauthorized: authentication required" │
Feb 21 13:59:18 machine dockerd[19533]: time="2018-02-21T13:59:18.079424439+01:00" level=error msg="Attempting next endpoint for push after error: unauthorized: authentication required"
I'm not 100% sure about it, but I think you use docker registry v1 and not v2. Is that right?
My next attempt was to checkout sudo journalctl -fu docker.service
.
The journal tells me:
Feb 21 14:04:01 machine dockerd[19533]: time="2018-02-21T14:04:01.118711156+01:00" level=warning msg="failed to retrieve docker-runc version: unknown output format: runc version spec: 1.0.0-rc2-dev
Feb 21 14:04:01 machine dockerd[19533]: "
Feb 21 14:04:01 machine dockerd[19533]: time="2018-02-21T14:04:01.118771754+01:00" level=warning msg="failed to retrieve docker-init version"
Feb 21 14:04:05 machine dockerd[19533]: time="2018-02-21T14:04:05.433738752+01:00" level=error msg="Upload failed: unauthorized: authentication required"
Feb 21 14:04:05 machine dockerd[19533]: time="2018-02-21T14:04:05.433843550+01:00" level=error msg="Attempting next endpoint for push after error: unauthorized: authentication required"
The logs are from right now. I check everything, while writing.
There were some colleagues helping me out with stuff, but we were unable to solve it. We even installed our own registry and there was no problem in pushing and pulling. The "docker-init version" and "docker-runc" messages also appeared. Even though I looked around in the internet to find a solution, I could not get rid of them. And the internet says that they are not relevant. Our own registry confirms that.
We also tried to do this stuff with packer, which has a docker-tag and docker-push hook. Btw packer is also unable to get a stable login. Also takes some tries. Anyway, the output is:
packer build otctest.json
docker output will be in this color.
==> docker: Creating a temporary directory for sharing data...
==> docker: Pulling Docker image: debian:stable-slim
docker: stable-slim: Pulling from library/debian
docker: Digest: sha256:79fa38298b2201284248246f51327403c8075d8c0704cf0e2694668d213d5b0f
docker: Status: Image is up to date for debian:stable-slim
==> docker: Starting docker container...
docker: Run command: docker run -v /home/me/.packer.d/tmp/packer-docker932334302:/packer-files -d -i -t debian:stable-slim /bin/bash
docker: Container ID: 37ec4b3083640b4d529e40b9dd9bb7136a20be0832fb0a5963c0b18d729f136c
==> docker: Committing the container
docker: Image ID: sha256:9bd3160e75df1441d12e5a0ccfb5bb4ab6013d868a59066523a0f9462d154a38
==> docker: Killing the container: 37ec4b3083640b4d529e40b9dd9bb7136a20be0832fb0a5963c0b18d729f136c
==> docker: Running post-processor: docker-tag
docker (docker-tag): Tagging image: sha256:9bd3160e75df1441d12e5a0ccfb5bb4ab6013d868a59066523a0f9462d154a38
docker (docker-tag): Repository: {ip}:{port}/something/otctest:latest
==> docker: Running post-processor: docker-push
docker (docker-push): Logging in...
docker (docker-push): Login Succeeded
docker (docker-push): Pushing: {ip}:{port}/something/otctest:latest
docker (docker-push): The push refers to a repository [{ip}:{port}/something/otctest]
docker (docker-push): e8f7eb100cd2: Preparing
docker (docker-push): ffb1014a0bfb: Preparing
docker (docker-push): unauthorized: authentication required
docker (docker-push): Logging out...
docker (docker-push): Removing login credentials for {ip}:{port}
Build 'docker' errored: 1 error(s) occurred:
* Post-processor failed: Bad exit status: 1
==> Some builds didn't complete successfully and had errors:
--> docker: 1 error(s) occurred:
* Post-processor failed: Bad exit status: 1
==> Builds finished but no artifacts were created.
Same thing here. The only thing I can think of, that is causing the failure, is
that the time on your server and the time on my machine are not identical, so the
authentication expires or somewhat. That was a problem that some people, which got
the same error messages, in the internet had with their registries. But I'm not able
to find out your server configuration. In the web interface we choosed eu-de
as our
location.
I really have no clue what to do now. Hopefully those information help you or someone to fix things. It would be nice if you could make things clear for us. Did we miss something or made something wrong?
Thank you in advance
Hi excurs0r,
I can see a couple of differences to my configuration. I have my daemon running with --insecure-registry
root 1376 1 0 2017 ? 01:09:47 /usr/bin/dockerd --log-level=info --insecure-registry 80.158.0.168:443 --containerd /run/containerd/containerd.sock
and I have a config.json wth the certificate info in it for authentication with the CCE registry.
cat ~/.docker/config.json {"auths":{"-SNIP-:443":{"auth":"XXXXXY-SNIP-XXXXXX==","email":""}}}
Have you downloaded the certificate from Registry/+Upload Container Image?
Regards
Anthony
Dear escurs0r,
at first the --insecure-registry must be configured properly in the docker host, which pulls or pushes images to the private (CCE) registry. I will try to explain it with an Ubuntu 16.04 host.
1) Add registry address
root@ecs-tino-ubuntu:~# cat /etc/docker/daemon.json
{
"insecure-registries": ["160.44.200.121:443"]
}
2) restart daemon
systemctl daemon-reload
service docker restart
3) download cert-file from CCE-Registry GUI and.. 3a)... copy the file to
root@ecs-tino-ubuntu:~# cat .docker/config.json
{
"auths": {
"160.44.200.121:443": {
"auth": "XYZ"
}
}
}
3b) ...or use the following command to get the _auth_token for the login process and login (safer way to check a correct registry login)
echo -n {auth} | base64 -d
docker login -u _auth_token -p {generated-token} 160.44.200.121:443
NOTE: The command -e aa {fancy-ip}:{some-port}
is wrongly described and will be fixed in the next documentation review.
4) Pull image, tag the image and upload it to the registry
root@ecs-tino-ubuntu:~# docker pull httpd
Using default tag: latest
latest: Pulling from library/httpd
4176fe04cefe: Pull complete
d6c01cf91b98: Pull complete
b7066921647a: Pull complete
643378aaba88: Pull complete
3c51f6dc6a3b: Pull complete
4f25e420c4cc: Pull complete
ccdbe37da15c: Pull complete
Digest: sha256:6e61d60e4142ea44e8e69b22f1e739d89e1dc8a2764182d7eecc83a5bb31181e
Status: Downloaded newer image for httpd:latest
root@ecs-tino-ubuntu:~# docker tag httpd 160.44.200.121:443/otc00000000001000000201/httpd:latest
root@ecs-tino-ubuntu:~# docker push 160.44.200.121:443/otc00000000001000000201/httpd:latest
The push refers to a repository [160.44.200.121:443/otc00000000001000000201/httpd]
11d3a23fba24: Pushed
6702ee5815dd: Pushed
ca92f217a68e: Pushed
400eca481024: Pushed
53cbc0080070: Pushed
a9681abc377f: Pushed
ffc4c11463ee: Pushed
latest: digest: sha256:6a457fe47eaa405ea173ca61d29c4367a593e8b092ed2e6c0fda0c77d801c485 size: 1780
Please let me know if it is working for you.
Kind regards, Tino
cat /etc/docker/daemon.json
{
"insecure-registries": ["160.44.200.121:443"]
}
cat .docker/config.json
{"auths":{"160.44.200.121:443":{"auth":"xxx","email":""}}}
sudo systemctl daemon-reload
sudo systemctl restart docker
docker login -u _auth_token -p xxx 160.44.200.121:443
Error response from daemon: Login: 404 page not found
(Code: 404; Headers: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff] Date:[Tue, 27 Feb 2018 11:21:07 GMT] Content-Length:[19]])
Login worked with old command:
docker login -u _auth_token -p xxx -e aa 160.44.200.121:443
Login Succeeded
[[ EDIT:
Retried docker login -u _auth_token -p xxx 160.44.200.121:443
And it worked. But still unable to get things running.
Compared login commands → No difference
]]
docker images | grep otc
160.44.200.121:443/otc00000000000000012345/otctest latest 9bfa9917d866 12 days ago 55.3MB
docker push 160.44.200.121:443/otc00000000000000012345/otctest
The push refers to a repository [160.44.200.121:443/otc00000000000000012345/otctest]
ffb1014a0bfb: Preparing
unauthorized: authentication require
The correct "username" for registry authentication is necessary. You need to use the correct tenant ID to pull images.
e.g. my Account name (tenant ID) is: OTC00000000001000000201 The correct docker image tag / pull needs to be
docker tag {imagename:tag} 160.44.200.121:443/otc00000000001000000201/{imagename:tag}
So: docker push 160.44.200.121:443/otc00000000000000012345/otctest will not work!
Please also check ~/.docker/config.json for your user, if the login information is placed properly after the login was succesfully done.
root@ecs-tino-ubuntu:~# cat .docker/config.json
{
"auths": {
"160.44.200.121:443": {
"auth": "X2F_shortened"
}
}
}
Hi there,
i am trying to deploy an infrastructure with docker containers to otc. I found several resources in the internet, but all i know so far is that it should be possible. It is like swimming in a sea of buzzwords.
An example of how to deploy a docker container with terraform to otc would help me out. Also may I am not the first person looking for this.
Could you please provide an example of how to deploy a docker container to otc via terraform?