Docker Hub registry: Add support for pulling images over IPv6

[thaJeztah]

Tell us about your request A clear and concise description of what you want to happen or the change you would like to see

As described in https://github.com/docker/hub-feedback/issues/1945, Docker Hub doesn't currently support pulling images over IPv6, which prevents users on an IPv6-only network from interacting with Docker Hub

Which service(s) is this request for?

Docker Hub registry

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?

See https://github.com/docker/hub-feedback/issues/1945

Are you currently working around the issue?

Users can use a NAT64 Gateway, as described in https://github.com/docker/hub-feedback/issues/1945#issuecomment-579519435

[xfgavin]

IPV6 should've been implemented already. Thank you.

[JuniperChris929]

Should have been implemented YEARS ago... Wake up! IPv6 is here - this is not optional anymore!

[pkennedyr]

Thanks all for the feedback - the Hub team is investigating the issue and will post an update as soon as we have more details to share.

In the meantime, if there are any special considerations / unique use cases in your environments that you would like taken into account, feel free to add your comments to this issue.

[JuniperChris929]

It's July now, almost August - any update on this?

[MaciejKucia]

Seems like Docker Hub does not work with AWS Egress-only internet gateway which is IPv6 by nature.

[JuniperChris929]

At this point it’s more than obvious that the Team is ignoring this probably biggest feature request and is actively and purposely trying to block the future. We are currently evaluating other solutions because without v6 Support Docker is useless for us. And we refuse to install 6-4 Gateways just because one team is too lazy (or looks for excuses) to deploy IPv6, which is the future. Ridiculous and sad...

[altelch]

RFC2460 is from december 1998...nearly 22 years later there are still services not available from legacy free networks...

[WilliamDEdwards]

Progress?

[PavelSosin-320]

Even enforced by law in some places! Even Providers fulfill it. nslookup aws.amazon.com Server: OpenWrt.lan Address: fd7d:e52e:3e3a::1

[WilliamDEdwards]

It's such a shame that "the world’s leading service for finding and sharing container images" is so ancient...

[jcyph3r]

Our cellphones pretty much all have IPv6 now. Google's worldwide traffic hits >33% IPv6 every week these days. Facebook's US traffic is >60% IPv6. Akamai's US traffic is nearly ~50% IPv6. Apple says IPv6 connection setup is 1.4x faster.

Please enable IPv6. IPv6 is here, and IPv6 is the future.

[aminvakil]

They're probably more concerned about setting limit on users who don't give money to the corporation... https://www.docker.com/blog/scaling-docker-to-serve-millions-more-developers-network-egress/

[Martin-Luther]

Please, when are you going to support IPv6 ? This is crazy !

[WilliamDEdwards]

Could anyone from Docker explain why Docker Hub still doesn't support IPv6? IPv6 is not that hard to implement. Sure, it's a bit more complex in a production environment - but that's still no reason not to support IPv6 in 2020! Does the Docker team lack the knowledge? Or does Docker simply not care about what's good for the internet as a whole?

It's products like Docker that make IPv6 implementation grind to a halt.

[miyurusankalpa]

Even the CDN endpoint (production.cloudflare.docker.com) which is powered by cloudflare does not have IPv6 enabled.

[PavelSosin-320]

Everybody can install the Registry Docker image locally on IPV6 enabled Docker engine. JFrog Container registry supports IPV6, obviously, enterprise infrastructure! Google Internet quality test reports lack IPV6 support as an error. My Router supports IPV6 as DNS and DHCP out-of-the-box. Lack of IPV6 support must be published as a restriction.

[WilliamDEdwards]

Everybody can install the Registry Docker image locally on IPV6 enabled Docker engine. JFrog Container registry supports IPV6, obviously, enterprise infrastructure! Google Internet quality test reports lack IPV6 support as an error. My Router supports IPV6 as DNS and DHCP out-of-the-box. Lack of IPV6 support must be published as a restriction.

In the time it takes to write "IPv6 is not supported at the moment", they could also implement IPv6.

[PavelSosin-320]

FYI: DockerHub uses AWS infrastructure: root@MSI:~# nslookup hub.docker.com root@MSI:~# nslookup hub.docker.com Server: 172.25.208.1 Address: 172.25.208.1 hub.docker.com canonical name = elb-default.us-east-1.aws.dckr.io. elb-default.us-east-1.aws.dckr.io canonical name = us-east-1-elbdefau-1nlhaqqbnj2z8-140214243.us-east-1.elb.amazonaws.com. Name: us-east-1-elbdefau-1nlhaqqbnj2z8-140214243.us-east-1.elb.amazonaws.com Address: 52.45.24.195 Name: us-east-1-elbdefau-1nlhaqqbnj2z8-140214243.us-east-1.elb.amazonaws.com Address: 34.206.208.51 Name: us-east-1-elbdefau-1nlhaqqbnj2z8-140214243.us-east-1.elb.amazonaws.com Address: 3.217.202.146 I According to This announcement regarding IPV6 support DockeHub should have no problem with it.

[ingshtrom]

Hey all,

Sorry for the delay! It is not acceptable to have this ticket sit so long without any response from Docker and I apologize for that.

As for the matter at hand, we would like to add support for this, but the complexity involved along with other tasks competing for our small team’s time does not make this feasible in the near term. This may change if there are more requests for this and/or internal priorities change. Keep in mind that there are a lot of moving parts involved with enabling IPv6 for us, along with the aforementioned competing, new work that is always being added. The implementation of this is not nearly as easy as flipping a switch or putting a new public-facing Application Load Balancer with dualstack support on the Internet.

For those of you requesting this, is there a specific reason you want or need this feature? Does IPv4 not work for you in some scenario or environment?

As others have already done, please add your :thumbsup: to the original issue comment so that we can track community interest in this.

[aminvakil]

@ingshtrom No, it does not. I have some virtual machines which does not have ipv4, right now I've set up squid on another virtual machine which has both ipv4 and ipv6 and make my requests through that.

[ingshtrom]

@ingshtrom No, it does not. I have some virtual machines which does not have ipv4, right now I've set up squid on another virtual machine which has both ipv4 and ipv6 and make my requests through that.

This is a naive question, but why can those virtual machines not have ipv4 on them?

[A1bi]

For those of you requesting this, is there a specific reason you want or need this feature? Does IPv4 not work for you in some scenario or environment?

For me the reason simply is the fact that for years now I have been deploying new systems without any IPv4 connectivity at all. It is just so much simpler. And most external services happily accept requests over IPv6, in my cases at least.

But in some cases I need to reach some IPv4-only services like Docker Hub for which I then have to implement transition mechanisms like NAT64+DNS64 and I absolutely want to get rid of this unnecessary complexity.

[aminvakil]

@ingshtrom No, it does not. I have some virtual machines which does not have ipv4, right now I've set up squid on another virtual machine which has both ipv4 and ipv6 and make my requests through that.

This is a naive question, but why can those virtual machines not have ipv4 on them?

Limitation of IPs on the hypervisor.

Edit: Every other thing is fine, they can act as web server behind cloudflare, it can retrieve OS packages from mirrors with ipv6, the only problem is docker hub.

[NoxInmortus]

@ingshtrom Thanks for your answer. As the original reporter of docker/hub-feedback#1945, I was waiting for an official feedback for 10 months...

From my point of view, we are already reaching the limits of IPv4 worldwide, and it's just a matter of time before we get "Nop, sorry, no more IPv4" from ISP and cloud providers.

In my user case, I wish to reach DockerHub with IPv6 connectivity because we will need to use IPv6 on the client side soon enough. And already, Cloud providers are charging every IPv4 (and they are right to do so), and give us a free /64 IPv6 network. What's the point to continue using IPv4 ? IPv6 is finalized since 1998, and everybody knew we were going to need it.

[altelch]

We often do pure IPv6 implementations if there is no need for IPv4 just to save address space. Everytime docker is required we need to do a dual stack setup just because of the docker repo and not because of the application requiring this...

[Martin-Luther]

For those of you requesting this, is there a specific reason you want or need this feature? Does IPv4 not work for you in some scenario or environment?

Self hosted Kubernetes + Docker on VMs without ( Cloud Providers) for all the benefits that you people must already know. Excepted for the load balancer, all of our kubernetes nodes will be IPv6 only in a near futur: for our use case at least.

How can I pull images from a source like Docker Hub/docker.io if my nodes are only talking IPv6 ? I have tried solutions like Tayga (nice by the way) + Bind9 for NAT64+DNS64, and it si only partially working. I am not talking about my docker images here, but those required by Kubernetes.

We are also a very small team and we have been running a dual-stack production app for 4 years now. So "being a small team" is not an excuse plus; you have by far, more money that my small business does.

Arrêtez de nous prendre pour des cons.

The implementation of this is not nearly as easy as flipping a switch or putting a new public-facing Application Load Balancer with dualstack support on the Internet.

Ok, but at least you could try to do something right ? Do it at least gradually and publish a list of your servers or apps that already support dual-stack. Slowly is still better than nothing. How many years did you have to complete this task ?

After this horrible waste of time, I think that I am just going to start to look for an alternative to replace the Docker engine by something else which is at least going to support IPv6 for images pulling. And the next person who is going to tell me how Docker is great will have my hand kissing is face.

[JuniperChris929]

What the f did I just read????? "does not make this feasible in the near term" --> GUYS - It's 2020 - WAKE UP!!!! (Insert Gordin Ramsay yelling)

IPv6 is no longer optional and having a small Team is no excuse at all. Heck, we implemented IPv6 for thousands of users across couple hundred servers with just 2 people for our colleagues worldwide. Sure it's taking time - but you could have started 20years ago...

Don't cry now because it's already too late and folks literally kick your balls --> you need to act NOW. You have put your head in the sand for way too long. If Docker seriously wants to wear "big boy pants" --> deploy IPv6 or prepare to be replaced.

Sick and tired of telling people why IPv6 is a must-have and IPv4 is dying - open your eyes - you can't be that blind... Why do we always have to show the same reasons over and over again... Jesus...

100.000 Networks, that hub.docker.com is not a part of: https://twitter.com/chsjuniper/status/1328486339172503552

-- Just my 2ct

[Gregy]

@ingshtrom I need to access docker images over IPv6 simply because I have no IPv4 address available. They are running out you know and their assignment is prioritized in the infrastructure I am using. There are simply none left for me.

[WilliamDEdwards]

This...

For those of you requesting this, is there a specific reason you want or need this feature? Does IPv4 not work for you in some scenario or environment?

...is the EXACT problem.

The 'specific reason we want or need this feature' doesn't matter. It's 2020. You're already 20 years late implementing IPv6.

Right now, the next generation of network admins is learning only IPv4, because hey, if Docker doesn't support IPv6, 'why should we'?

Let's not pretend Docker is a limited, small team of people who don't have enough money to implement IPv6. You can have the resources if you want to have the resources. But it seems like the Docker team is still stuck in the 'why'. We have surpassed the stage where we have to think about why we should implement IPv6. For the past 20 years, IPv6 has been a fact of life!

Businesses like Docker are the very reason IPv6 is where it is today.

[certuna]

Dual stack networks is double the admin work, nobody likes it.

Because of this, admins increasingly skip this step and go directly from single stack IPv4 to single stack IPv6 networks, with IPv4-as-a-service on top (DNS64+NAT64). Docker has to work in a single-stack IPv6 environment, or else it ends up being the only legacy application holding back upgrades.

[WilliamDEdwards]

it ends up being the only legacy application holding back upgrades.

It already is.

Not exactly what you'd expect from a business that positions itself as 'empowering developers'. It holds them back.

[archoversight]

For those of you requesting this, is there a specific reason you want or need this feature? Does IPv4 not work for you in some scenario or environment?

Certain VPS providers are now charging extra per month for IPv4 IP addresses, or are not providing IPv4 addressing at all. Also users in certain businesses that don't want to pay extra for a static public IP are being dropped behind CG-NAT installations which doesn't play well with the Dockhub limitations on pulls.

[unquietwiki]

I use Linode right now for stuff, and they're IPv6 native + shared IPv4 in their environment. I've also run into more than one occasion where IPv6 worked better than IPv4 in a technical difficulty situation. I also found out about this via https://www.reddit.com/r/ipv6/comments/jvrhl5/docker_engineers_are_asking_for_specific_reason/ , and as an moderator there, I wholly support this move!

[binman-docker]

Hi, I just want to remind some contributors to please keep things professional. This is important, yes, and it will happen. But language that is borderline abusive doesn't make it happen any faster, especially when that language is directed at the folks who will be doing the work and who are trying to engage with you rather than ignore the issue.

I also want to be very clear that Docker is a small company and work has to be prioritized. The infra team is four people - people dealing with a long legacy of the way Hub and Registry itself were built. Hub services millions of users doing billions of pulls every week, so changes made to our core stack are not made lightly. The sheer scale of Hub and the systems powering it means that this isn't just flipping a switch or deploying a new load balancer. There is risk involved in any change like this and it will be done slowly and methodically.

All that said, right now the plan is for this to go onto the 2021 roadmap - the earlier the better, but I can't make any specific promises at this point. If anyone has feedback on implementation or user experience they would like to suggest, we'd love to hear it. Thank you to those who have already offered your scenarios and constructive feedback to help us understand your needs.

[unquietwiki]

@binman-docker is there a way we as a community can help make this change?

[Martin-Luther]

@binman-docker This issue is addressed to the people who make decisions, not to the folks who are going to implement the solution. I think they did not even need this issue to understand how important it is for all of us, and how frustrating this has become with years now.

All that said, right now the plan is for this to go onto the 2021 roadmap - the earlier the better, but I can't make any specific promises at this point.

Is it a public announcement ?

If anyone has feedback on implementation or user experience they would like to suggest, we'd love to hear it.

You know that most of us here will be more than happy to help, but we have to know what are your technical issues first. How do we proceed if we want to help ?

[telmich]

Good morning dev team,

For those of you requesting this, is there a specific reason you want or need this feature? Does IPv4 not work for you in some scenario or environment?

As others have already done, please add your +1 to the original issue comment so that we can track community interest in this.

it's very simple: more and more networks (all that we use here in the Glarus mountains) are IPv6 only. If we want to continue using docker, docker needs to work IPv6 only networks.

[WilliamDEdwards]

Hi, I just want to remind some contributors to please keep things professional. This is important, yes, and it will happen. But language that is borderline abusive doesn't make it happen any faster, especially when that language is directed at the folks who will be doing the work and who are trying to engage with you rather than ignore the issue.

I agree that things should be kept professional.

But let's be honest, it's not completely true that Docker is actively trying to engage with the community. This issue has been idle for months with no reply from Docker at all. For something as important as IPv6, that's extremely frustrating. Like your colleague said: "It is not acceptable to have this ticket sit so long without any response from Docker."

The question "Does IPv4 not work for you in some scenario or environment?" shows the Docker team does not understand how important IPv6 is. This question implies IPv4 should be the first choice, but it never should be. That's where this frustration stems from.

[telmich]

@binman-docker Regarding your previous comment about the infra team: we are building IPv6 infrastructures for customers on a daily basis and enabling the hub with IPv6 seems to be a rather easy thing to solve for me. In case you are interested in commercial support getting the hub running with IPv6, give me a ping at ipv6 at ungleich.ch.

[sep76]

The question "Does IPv4 not work for you in some scenario or environment?" shows the Docker team does not understand how important IPv6 is. This question implies IPv4 should be the first choice, but it never should be. That's where this frustration stems from.

This is so important to understand. everything nowadays should be ipv6 only by default. ipv6 with dns64/nat64 is a workaround if something do not work with ipv6 only, ipv6+ipv4 dual stack is a workaround if $something do not work with ipv6+dns64. and running ipv4 only is just not an option at all, except for limited very legacy applications.
Docker hub registry not supporting ipv6 lead to a lot of people having to implement workarounds, on their whole infrastructure.

[jhmartin]

Another aspect of IPv6 is that there are so many IPs that the hub's current ip-based rate-limiting mechanism would be rendered useless -- clients could use a different ip for every image pull and never reuse one.

[telmich]

@jhmartin That is easy to solve by matching on /64 or /48.

[WilliamDEdwards]

Another aspect of IPv6 is that there are so many IPs that the hub's current ip-based rate-limiting mechanism would be rendered useless -- clients could use a different ip for every image pull and never reuse one.

Just rate limit the /64 ...

[chrisnew]

I’m also in favor of having an IPv6 enabled hub infrastructure. ELB started to support IPv6 for NLBs recently.

[sep76]

Another aspect of IPv6 is that there are so many IPs that the hub's current ip-based rate-limiting mechanism would be rendered useless -- clients could use a different ip for every image pull and never reuse one.

even if you do the quick and dirty fix of putting a ipv6 revers proxy in front of the ipv4 service, it would work. The ipv4 service will in effect rate limit all the ipv6 users from that proxy as one, but as long as you can tune the rate limit, and have a few globally spread reverse proxies, it would function.
Similar to how you deal with CGN today, where thousands of internet users are behind a single pool of public ipv4 address.

[PavelSosin-320]

@sep76 You are right 100% The stable IP address and MAC are in the past. I can refresh both IPV4 and IPV6 addresses of all my devices at home by single button touch on my router box or its GUI. Still statefull session base communication is possible via cookies-likes key interchange with expiration like Chrome with Kubernetes do it. DockerHub also will never see my laptop IP because I connected to the provider's WAN via WiFi router with dynamic IPV6 address. In the area where I leave IPV6 support is mandatory for providers.

[telmich]

I think there is an easy TL;DR here:

• IPv6 has become a standard and IPv6 only networks are becoming standard
• Any software that does not fully support IPv6 only networks has and will more become legacy
• If docker wants to be accepted by the IPv6 community and continue to work in the foreseeable future, all infrastructure, including the hub needs to be IPv6 compatible
• The IPv6 community is very open to discuss the question of "how" to implement this
• However there is no room for "if" discussions anymore, we are all past this point for way too long.

[WilliamDEdwards]

Any updates?

[seimnseimn]

Is there an open position as CTO here? Because I'm pretty sure this position is currently not filled?

[serverwentdown]

I very much agree that we needed IPv6 support since forever, but I do think everyone should stop throwing shade here. Your energy is better spent being a positive advocate for IPv6 elsewhere.