Closed thaJeztah closed 2 years ago
I very much agree that we needed IPv6 support since forever, but I do think everyone should stop throwing shade here. Your energy is better spent being a positive advocate for IPv6 elsewhere.
Agree throwing shade is no good, but not sure we should let this issue be idle for ages again.
Docker Hub is a free service so unless you are paying, there is no reason to expect anything.
AWS will open public registry soon. Let's see what they come up with.
@MaciejKucia
Docker Hub is a free service
I would not be so sure about that. You have to pay in order to avoid pulling limits. Even when you pay, you are still stuck with the lack of support for IPv6. This is not normal in 2020. Amazon ECR is already up and running (https://aws.amazon.com/ecr/), and something is telling me that they did not make the same mistake. The problem is that Docker Hub has been really popular these last years and a lot of projects were relying on their registry. So, if for your work you are relying on one more projects hosting the required images on Docker Hub and you also need support for IPv6 (pulling) because this is the way your infra was built, you are screwed: unless you are ready to downgrade or go with dual-stack :
I am sorry if it hurts, but the more time they will wait, the more people will be put in miserable situations like this.
Let's see what they come up with.
I hope that decisions makers at Docker Hub will wake up. Today, it is not like we had the choice. Yeah, Let's see what Amazon come up with, but today you still have to pay to host your docker images: if I am wrong, I apologise.
Docker Hub is a free service so unless you are paying, there is no reason to expect anything.
You're right, we shouldn't expect anything. Lack of IPv6 support just goes to show that Docker Hub's commercial interests exceed what's good for the internet as a whole. I have an issue with that.
👋 hiya!
Any updates?
A very small one, but hopefully it helps you understand that we do care. This has been on my mind almost every day for the last two weeks. I have felt frustrated and attacked but then I also felt a motivation to research and understand more. While others on the Docker Infrastructure Team have more knowledge about IPv6 implementations, my knowledge is limited and I have been doing research over the same time to become more acquainted with the technology.
@ingshtrom The approach seems to be valid. In general when moving from IPv4 towards IPv6 most projects don't do a big bang. Overall I can read from your post quite a good strategy, but I'll rephrase it to make it a bit more clear:
As you said, this is basically updating the incoming proxies, parsing code and eventually the domain names.
While it is totally feasible not to migrate your whole infrastructure to IPv6 right away, I strongly suggest that from the moment that the hub is productively reachable via IPv6, you introduce an IPv6 first policy for new services. This has not only the advantage of not repeating previous problems, but you actually do not need to support IPv4 anymore: the IPv4 Internet is easily mapped to a tiny /96 IPv6 networks.
From this moment on, we and yourself can welcome you in the new Internet - things will get dramatically easier for developing and maintenance. It will be a longer way, but from our experience it is worth the reduction of complexity.
In case you do need some support for the migration, I assume many of us here are willing to help. As a matter of fact there is even an open IPv6 chat at https://IPv6.chat. From my side I'm working for ungleich who is also offering commercial IPv6 support. Contact me at ipv6 at ungleich.ch if that is of interest for the migration.
TL;DR
Thanks for the details, let's move forward.
While others on the Docker Infrastructure Team have more knowledge about IPv6 implementations, my knowledge is limited and I have been doing research over the same time to become more acquainted with the technology.
Good to hear. When will the Docker Infrastructure Team start preparations?
@WilliamDEdwards the next two months will include work that moves us towards IPv6 support. I cannot say that you will notice anything, but since the whole thing can be done piecemeal, we can chip it off as we accomplish other work.
I ran into this problem too. This is what I do to allow me pulling image from docker hub.
Set up a HTTP proxy in another dual stack VPS. I use goproxy. https://github.com/snail007/goproxy/releases Start proxy, it listens on 33080 by default. $ ./proxy http --debug
In the docker host, modify the env for systemd. $ sudo systemctl edit docker.service Add the following to the file.
[Service]
Environment="HTTP_PROXY=http://[Dualstack_IPv6]:33080"
Environment="HTTPS_PROXY=https://[Dualstack_IPv6]:33080"
$ sudo systemctl daemon-reload
$ sudo systemctl restart docker
$ docker pull nginx
"implementation of this is not nearly as easy as flipping a switch or putting a new public-facing Application Load Balancer with dualstack support on the Internet."
If a DNS64 + NAT64 workaround works from the client side, then yes, it should be as simple as adding a new IPv6 address to the machines, or reverse proxies, at your side.
Because that is what DNS64 effectively does -- give the machines an IPv6 address (a local, special address though).
registry-1.docker.io already has a half dozen addresses (A records); just add some IPv6. If you can't do dual stack for your servers, than a reverse proxy can be easily set up; or get a CDN to do it for you. A simple reverse proxy or CDN is probably the easiest/simplest way to get IPv6.
There are also a few offers of IPv6 help above, e.g. ungleich.ch, who have said they are happy to help.
DNS record of web-interface, hub.docker.com
, points to elastic balancer in Amazon cloud:
% host hub.docker.com
hub.docker.com is an alias for elb-default.us-east-1.aws.dckr.io.
elb-default.us-east-1.aws.dckr.io is an alias for us-east-1-elbdefau-1nlhaqqbnj2z8-140214243.us-east-1.elb.amazonaws.com.
us-east-1-elbdefau-1nlhaqqbnj2z8-140214243.us-east-1.elb.amazonaws.com has address 3.217.202.146
us-east-1-elbdefau-1nlhaqqbnj2z8-140214243.us-east-1.elb.amazonaws.com has address 34.206.208.51
us-east-1-elbdefau-1nlhaqqbnj2z8-140214243.us-east-1.elb.amazonaws.com has address 52.45.24.195
which already can work with dual-stack (just add dualstack
to DNS record):
% host dualstack.us-east-1-elbdefau-1nlhaqqbnj2z8-140214243.us-east-1.elb.amazonaws.com.
dualstack.us-east-1-elbdefau-1nlhaqqbnj2z8-140214243.us-east-1.elb.amazonaws.com has address 52.45.24.195
dualstack.us-east-1-elbdefau-1nlhaqqbnj2z8-140214243.us-east-1.elb.amazonaws.com has address 3.217.202.146
dualstack.us-east-1-elbdefau-1nlhaqqbnj2z8-140214243.us-east-1.elb.amazonaws.com has address 34.206.208.51
dualstack.us-east-1-elbdefau-1nlhaqqbnj2z8-140214243.us-east-1.elb.amazonaws.com has IPv6 address 2406:da00:ff00::342d:18c3
dualstack.us-east-1-elbdefau-1nlhaqqbnj2z8-140214243.us-east-1.elb.amazonaws.com has IPv6 address 2406:da00:ff00::22ce:d033
dualstack.us-east-1-elbdefau-1nlhaqqbnj2z8-140214243.us-east-1.elb.amazonaws.com has IPv6 address 2406:da00:ff00::3d9:ca92
But something is not configured on ELB side:
% curl -vI --connect-to ::dualstack.us-east-1-elbdefau-1nlhaqqbnj2z8-140214243.us-east-1.elb.amazonaws.com:443 "https://hub.docker.com/_/python"
* Connecting to hostname: dualstack.us-east-1-elbdefau-1nlhaqqbnj2z8-140214243.us-east-1.elb.amazonaws.com
* Connecting to port: 443
…
* Trying 2406:da00:ff00::3d9:ca92...
* TCP_NODELAY set
* Expire in 149989 ms for 3 (transfer 0x56505cfdaec0)
* Expire in 200 ms for 4 (transfer 0x56505cfdaec0)
* connect to 2406:da00:ff00::3d9:ca92 port 443 failed: Connection refused
* Trying 2406:da00:ff00::22ce:d033...
* TCP_NODELAY set
* Expire in 149938 ms for 3 (transfer 0x56505cfdaec0)
* connect to 2406:da00:ff00::22ce:d033 port 443 failed: Connection refused
* Trying 2406:da00:ff00::342d:18c3...
* TCP_NODELAY set
* Expire in 149883 ms for 3 (transfer 0x56505cfdaec0)
* connect to 2406:da00:ff00::342d:18c3 port 443 failed: Connection refused
* Trying 34.206.208.51...
* TCP_NODELAY set
* Expire in 149831 ms for 3 (transfer 0x56505cfdaec0)
* Connected to dualstack.us-east-1-elbdefau-1nlhaqqbnj2z8-140214243.us-east-1.elb.amazonaws.com (34.206.208.51) port 443 (#0)
…
My ISP doesn't want to wait for DockerHub decision and sent me the announcement: PartnerISP_IPV6_Announcement.pdf. This ISP serves 1/2 of customers in Silicon waddy.
What are the current free Docker registry on the market that support IPv6? I saw AWS ECR, but I don't find other ones. Can someone post a list of them?
@WilliamDEdwards the next two months will include work that moves us towards IPv6 support. I cannot say that you will notice anything, but since the whole thing can be done piecemeal, we can chip it off as we accomplish other work.
It's 2 months later :). Any progress?
What are the current free Docker registry on the market that support IPv6? I saw AWS ECR, but I don't find other ones. Can someone post a list of them?
Here is a list I collected a while back, only Google Container registry had v6 support. Also note that some registries point to other CDN domains to deliver the image files, which may have IPv6 enabled. | Domain | IPv6 |
---|---|---|
gcr.io | Y | |
registry-1.docker.io | N | |
ghcr.io | N | |
registry.gitlab.com | N | |
rg.nl-ams.scw.cloud | N | |
registry.digitalocean.com | N | |
myregistry.azurecr.io | N | |
137112412989.dkr.ecr.us-east-1.amazonaws.com | N | |
ecr-public.us-east-1.amazonaws.com | N | |
registry.access.redhat.com | N | |
registry.redhat.io | N | |
registry.connect.redhat.com | N |
It's 2 months later :). Any progress?
What do you know, time flies! We are actively working on a necessary piece of work involving our routing infra, but have not made direct progress on ipv6 due to priorities. The work being done right now is an absolute requirement in order to support ipv6 and we will definitely keep this thread updated.
Amazing.
Anyone suggest some ipv6 mirrors ?
@IMEVER , Google Cloud has a Docker hub mirror: mirror.gcr.io. It has IPv6 address:
$ host mirror.gcr.io
mirror.gcr.io is an alias for googlecode.l.googleusercontent.com.
googlecode.l.googleusercontent.com has address 108.177.119.82
googlecode.l.googleusercontent.com has IPv6 address 2a00:1450:4013:c00::52
Here is a documentation how to use it: Pulling cached Docker Hub images.
For example, add into /etc/docker/daemon.json
:
{
"registry-mirrors": ["https://mirror.gcr.io"]
}
@IMEVER , Google Cloud has a Docker hub mirror: mirror.gcr.io. It has IPv6 address:
$ host mirror.gcr.io mirror.gcr.io is an alias for googlecode.l.googleusercontent.com. googlecode.l.googleusercontent.com has address 108.177.119.82 googlecode.l.googleusercontent.com has IPv6 address 2a00:1450:4013:c00::52
Here is a documentation how to use it: Pulling cached Docker Hub images. For example, add into
/etc/docker/daemon.json
:{ "registry-mirrors": ["https://mirror.gcr.io"] }
Thanks, now I can pull image.
But when I use docker search xxx
, it still goes to docker.io, any idea ?
[root@vultr ~]# docker search nginx
Error response from daemon: Get https://index.docker.io/v1/search?q=nginx&n=25: dial tcp 34.195.201.174:443: connect: network is unreachable
@IMEVER I don't think the stuff that docker search
uses was ever a part of the OCI distribution spec or the distribution. It is something that is a value add on built into the Docker CLI which works with Docker Hub.
nebuk89 moved this from Investigating to We're Writing The Code in docker-roadmap
6 months later, at last!
@IMEVER I don't think the stuff that
docker search
uses was ever a part of the OCI distribution spec or the distribution. It is something that is a value add on built into the Docker CLI which works with Docker Hub.
search was part of the v1 docker registry (https://github.com/docker-archive/docker-registry), but was explicitly left out of the v2 docker registry specs (which was donated to the OCI to become the OCI distribution spec)
Search was left out of the spec, because it's orthogonal to "distributing" images, and different registries had different requirements for what they would offer w.r.t. "search" (if any).
So, although it would "technically" be possible to have docker search
also switch to use the configured mirror, it would break in most (if not all) setups, unless the mirror would also provide the combination of v1 and v2 API's (not sure if there's implementations that do this).
In any case, that would be a feature request for https://github.com/moby/moby, and for the maintainers of the Moby project to decide.
it is still missing...
# docker run hello-world
Unable to find image 'hello-world:latest' locally
docker: Error response from daemon: Get "https://registry-1.docker.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers).
See 'docker run --help'.
only legacy IP adresses:
# host registry-1.docker.io
registry-1.docker.io has address 54.85.56.253
registry-1.docker.io has address 52.72.232.213
registry-1.docker.io has address 54.161.109.204
registry-1.docker.io has address 34.198.213.42
registry-1.docker.io has address 52.20.81.6
registry-1.docker.io has address 34.238.187.50
registry-1.docker.io has address 34.192.204.44
registry-1.docker.io has address 3.209.182.229
$ host registry-1.docker.io
registry-1.docker.io has address 107.23.149.57
registry-1.docker.io has address 54.161.109.204
registry-1.docker.io has address 34.192.204.44
registry-1.docker.io has address 52.72.232.213
registry-1.docker.io has address 3.213.204.48
registry-1.docker.io has address 54.236.165.68
registry-1.docker.io has address 34.192.145.113
registry-1.docker.io has address 3.229.227.53
registry-1.docker.io has IPv6 address 2a0a:e5c0:0:1::6b17:9539
registry-1.docker.io has IPv6 address 2a0a:e5c0:0:1::3d5:cc30
registry-1.docker.io has IPv6 address 2a0a:e5c0:0:1::3e5:e335
registry-1.docker.io has IPv6 address 2a0a:e5c0:0:1::3448:e8d5
registry-1.docker.io has IPv6 address 2a0a:e5c0:0:1::22c0:9171
registry-1.docker.io has IPv6 address 2a0a:e5c0:0:1::22c0:cc2c
registry-1.docker.io has IPv6 address 2a0a:e5c0:0:1::36a1:6dcc
registry-1.docker.io has IPv6 address 2a0a:e5c0:0:1::36ec:a544
🤷
$ host registry-1.docker.io registry-1.docker.io has address 107.23.149.57 registry-1.docker.io has address 54.161.109.204 registry-1.docker.io has address 34.192.204.44 registry-1.docker.io has address 52.72.232.213 registry-1.docker.io has address 3.213.204.48 registry-1.docker.io has address 54.236.165.68 registry-1.docker.io has address 34.192.145.113 registry-1.docker.io has address 3.229.227.53 registry-1.docker.io has IPv6 address 2a0a:e5c0:0:1::6b17:9539 registry-1.docker.io has IPv6 address 2a0a:e5c0:0:1::3d5:cc30 registry-1.docker.io has IPv6 address 2a0a:e5c0:0:1::3e5:e335 registry-1.docker.io has IPv6 address 2a0a:e5c0:0:1::3448:e8d5 registry-1.docker.io has IPv6 address 2a0a:e5c0:0:1::22c0:9171 registry-1.docker.io has IPv6 address 2a0a:e5c0:0:1::22c0:cc2c registry-1.docker.io has IPv6 address 2a0a:e5c0:0:1::36a1:6dcc registry-1.docker.io has IPv6 address 2a0a:e5c0:0:1::36ec:a544
🤷
These are nat64 addresses, not the one from docker company.
Yeah, regstry-1.docker.io is still IPv4 only.
First query is from my local machine (dual stack); I also threw in a check of the mirror (that does have IPv6). Second query is from a host I have that is IPv6 only; it has DNS64+NAT64 configured for outgoing (to get around IPv4 only sites).
PS /home/sly> host registry-1.docker.io
registry-1.docker.io has address 54.85.56.253
registry-1.docker.io has address 3.213.204.48
registry-1.docker.io has address 3.226.210.61
registry-1.docker.io has address 52.55.168.20
registry-1.docker.io has address 107.23.149.57
registry-1.docker.io has address 54.236.165.68
registry-1.docker.io has address 52.204.76.244
registry-1.docker.io has address 54.161.109.204
PS /home/sly> host mirror.gcr.io
mirror.gcr.io is an alias for googlecode.l.googleusercontent.com.
googlecode.l.googleusercontent.com has address 172.217.194.82
googlecode.l.googleusercontent.com has IPv6 address 2404:6800:4003:c04::52
PS /home/sly> ssh root@gryphon001.vs.mythic-beasts.com
Welcome to Ubuntu 20.04.3 LTS (GNU/Linux 5.4.0-84-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
Last login: Tue Sep 21 10:02:41 2021 from 2407:8800:bc61:1300:2d48:d479:af90:c61f
root@gryphon001:~# host registry-1.docker.io
registry-1.docker.io has address 34.192.145.113
registry-1.docker.io has address 3.213.204.48
registry-1.docker.io has address 34.192.204.44
registry-1.docker.io has address 3.229.227.53
registry-1.docker.io has address 3.209.182.229
registry-1.docker.io has address 54.85.56.253
registry-1.docker.io has address 3.226.210.61
registry-1.docker.io has address 52.72.232.213
registry-1.docker.io has IPv6 address 64:ff9b::3448:e8d5
registry-1.docker.io has IPv6 address 64:ff9b::3e2:d23d
registry-1.docker.io has IPv6 address 64:ff9b::3d1:b6e5
registry-1.docker.io has IPv6 address 64:ff9b::22c0:cc2c
registry-1.docker.io has IPv6 address 64:ff9b::3655:38fd
registry-1.docker.io has IPv6 address 64:ff9b::3d5:cc30
registry-1.docker.io has IPv6 address 64:ff9b::3e5:e335
registry-1.docker.io has IPv6 address 64:ff9b::22c0:9171
root@gryphon001:~#
Hello everyone - I'm really happy to announce that we now have a beta IPv6 endpoint for Docker Hub Registry!
You can read some of the details (including implementation) on the blog: https://www.docker.com/blog/beta-ipv6-support-on-docker-hub-registry/
And please use this issue for feedback and technical discussion: https://github.com/docker/hub-feedback/issues/2165
I want to note upfront that this is currently BETA - there is no guarantee of functionality, uptime, or that it will exist in the future. Please only use this for testing. What happens in the future (keeping this domain, making the main domain dualstack, etc) depends on the feedback we receive, so please let us know how it goes!
Awesome! I will definitely try it out. That blog article perfectly explains why this wasn’t as easy as just flipping a switch. Great work!
@binman-docker Since the current implementation is a BETA feature and not recommended for production, do you still in mind that this Feature is really shipped?
I personally see this issues still in 'Developer Preview/Experimental' and am interesting into a Roadmap item, when IPv6 is officially available. In terms of officially, I speak about the registry-1.docker.io
endpoint.
@binman-docker Since the current implementation is a BETA feature and not recommended for production
Just cross posting for the sake of the thread, that the IPv6-only endpoint was apparently promoted out of beta: https://github.com/docker/hub-feedback/issues/2165#issuecomment-1020386475
I'd still agree as another user that dual stacking the primary endpoint is what would make this "shipped"
;()-[-}
2023 not yet?
@aliel It seems like registry.docker.com
now both answers with IPv4/IPv6 in dual stack, so I assume this is fixed, otherwise I would have complained =)
Can confirm:
$ host registry.docker.com
registry.docker.com has address 52.1.184.176
registry.docker.com has address 34.194.164.123
registry.docker.com has address 18.215.138.58
registry.docker.com has IPv6 address 2600:1f18:2148:bc01:a3b0:6734:c617:7c5c
registry.docker.com has IPv6 address 2600:1f18:2148:bc00:8334:ca86:c3d6:a507
registry.docker.com has IPv6 address 2600:1f18:2148:bc02:cfd8:db68:ea1f:277c
Hi everyone, just wanted to update here that we have officially rolled out dual-stack networking capabilities across all of our Docker Hub Registry, Docker Docs, and Docker Scout endpoints!
Read more about it here: https://www.docker.com/blog/docker-hub-registry-ipv6-support-now-generally-available/
Tell us about your request A clear and concise description of what you want to happen or the change you would like to see
As described in https://github.com/docker/hub-feedback/issues/1945, Docker Hub doesn't currently support pulling images over IPv6, which prevents users on an IPv6-only network from interacting with Docker Hub
Which service(s) is this request for?
Docker Hub registry
Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
See https://github.com/docker/hub-feedback/issues/1945
Are you currently working around the issue?
Users can use a NAT64 Gateway, as described in https://github.com/docker/hub-feedback/issues/1945#issuecomment-579519435