I am not able to pull an image from docker registry to upgrade my kubernetes cluster version to v1.28.10 in my production server. This is the error message when it's trying to pull the image:
Failed to pull image "rancher/kubectl:v1.28.2": rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/rancher/kubectl:v1.28.2": failed to resolve reference "docker.io/rancher/kubectl:v1.28.2": failed to do request: Head "https://registry-1.docker.io/v2/rancher/kubectl/manifests/v1.28.2": read tcp 9.144.23.87:54336->3.219.239.5:443: read: connection reset by peer
1. Manual Pull
When I try to login to the server and do a manual pull with the command "docker pull rancher/kubectl:v1.28.2" I get the following error:
"Error response from daemon: Get "https://registry-1.docker.io/v2/": read tcp 9.144.23.87:54336->3.219.239.5:443: read: connection reset by peer"
This only happens in the Dalian/China site , hence I checked with firewall team but the traffic is already allowed on their end according to their firewall logs.
2. Firewall
Upon checking with the site firewall team, the connectivity and traffic is working with telnet but when pulling the image it shows a connection reset by the destination(docker registry), I already requested the firewall team to allow any to any rule for connectivity between source and destination and vice versa but it still doesn't work.
3. Differences
The image pull works on other sites: West Europe, East US, South East Asia etc
However it doesn't work in China/Dalian site. The last successful image pull and kubernetes upgrade for this site in Dalian city was on the last week of August. (26th - 30th August 2024)
Aside from that the set up and steps to pull the image and upgrade the kubernetes cluster version is the same as the other sites.
Question
Have there been any changes by docker community in September that blocks the request/connection to the docker registry from China/Dalian(It was last working at 26th - 30th August 2024) ?
Did anyone face similar issues(connection reset at peer) trying to pull the image from docker registry in a server/VM at China/Dalian?
If this is a docker registry issue, how can I reach out to the docker registry team to check further on why the connection is resetting on docker registry end?
Thank You.
Any other ideas or suggestions aside from the above questions are also greatly appreciated.
Problem
I am not able to pull an image from docker registry to upgrade my kubernetes cluster version to v1.28.10 in my production server. This is the error message when it's trying to pull the image:
Failed to pull image "rancher/kubectl:v1.28.2": rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/rancher/kubectl:v1.28.2": failed to resolve reference "docker.io/rancher/kubectl:v1.28.2": failed to do request: Head "https://registry-1.docker.io/v2/rancher/kubectl/manifests/v1.28.2": read tcp 9.144.23.87:54336->3.219.239.5:443: read: connection reset by peer
1. Manual Pull When I try to login to the server and do a manual pull with the command "docker pull rancher/kubectl:v1.28.2" I get the following error:
"Error response from daemon: Get "https://registry-1.docker.io/v2/": read tcp 9.144.23.87:54336->3.219.239.5:443: read: connection reset by peer"
This only happens in the Dalian/China site , hence I checked with firewall team but the traffic is already allowed on their end according to their firewall logs.
2. Firewall Upon checking with the site firewall team, the connectivity and traffic is working with telnet but when pulling the image it shows a connection reset by the destination(docker registry), I already requested the firewall team to allow any to any rule for connectivity between source and destination and vice versa but it still doesn't work.
3. Differences The image pull works on other sites: West Europe, East US, South East Asia etc
However it doesn't work in China/Dalian site. The last successful image pull and kubernetes upgrade for this site in Dalian city was on the last week of August. (26th - 30th August 2024)
Aside from that the set up and steps to pull the image and upgrade the kubernetes cluster version is the same as the other sites.
Question
Have there been any changes by docker community in September that blocks the request/connection to the docker registry from China/Dalian(It was last working at 26th - 30th August 2024) ?
Did anyone face similar issues(connection reset at peer) trying to pull the image from docker registry in a server/VM at China/Dalian?
If this is a docker registry issue, how can I reach out to the docker registry team to check further on why the connection is resetting on docker registry end?
Thank You.
Any other ideas or suggestions aside from the above questions are also greatly appreciated.