search

docker / for-win

Bug reports for Docker Desktop for Windows
https://www.docker.com/products/docker#/windows
1.86k stars 289 forks source link

[Docker Desktop for Windows] Docker Engine v20.10.12 no longer working with proxy #12503

Closed mhoare1984 closed 2 years ago

mhoare1984 commented 2 years ago

Actual behavior

docker pull mcr.microsoft.com/dotnet/sdk:6.0 Error response from daemon: Get "https://mcr.microsoft.com/v2/": dial tcp 204.79.197.219:443: connect: connection refused

Expected behavior

A successful pull

Information

Docker Engine v20.10.12 on Windows via Docker Desktop 4.4.X no longer works with our corporate proxy. I can no longer do any docker command that connects to the internet. Two other people in our office are experiencing the same issue. If we uninstall Docker Desktop 4.4.X and reinstall 4.3.2 which comes with Docker Engine v20.10.11, the issue is resolved.

Our corporate proxy is Squid. we have set the HTTP and HTTP docker proxy settings to http://ourproxy.domain:3128. The proxy only has a HTTP endpoint so we set the HTTP and HTTPS proxy setting to the same value.

Output of & "C:\Program Files\Docker\Docker\resources\com.docker.diagnose.exe" check

[020:15:39:02.952][I] ipc.NewClient: 31a8fce3-com.docker.diagnose -> \.\pipe\dockerBackendV2 com.docker.service.exe [goroutine 1 [running, locked to thread]: [win/backend/pkg/service.NewClientForPath(...) [ win/backend/pkg/service/service.go:31 [win/backend/pkg/service.NewClient({0xb0d772, 0x13}) [ win/backend/pkg/service/service.go:25 +0xc5 [common/pkg/diagkit/gather/diagnose.init() [ common/pkg/diagkit/gather/diagnose/ipc_windows.go:23 +0x118 Starting diagnostics

[PASS] DD0027: is there available disk space on the host? [PASS] DD0028: is there available VM disk space? [PASS] DD0031: does the Docker API work? [PASS] DD0004: is the Docker engine running? [PASS] DD0011: are the LinuxKit services running? [PASS] DD0016: is the LinuxKit VM running? [PASS] DD0001: is the application running? [SKIP] DD0018: does the host support virtualization? [PASS] DD0002: does the bootloader have virtualization enabled? [PASS] DD0017: can a VM be started? [PASS] DD0024: is WSL installed? [PASS] DD0021: is the WSL 2 Windows Feature enabled? [PASS] DD0022: is the Virtual Machine Platform Windows Feature enabled? [PASS] DD0025: are WSL distros installed? [PASS] DD0026: is the WSL LxssManager service running? [PASS] DD0029: is the WSL 2 Linux filesystem corrupt? [PASS] DD0015: are the binary symlinks installed? [PASS] DD0003: is the Docker CLI working? [PASS] DD0013: is the $PATH ok? [PASS] DD0005: is the user in the docker-users group? [PASS] DD0007: is the backend responding? [PASS] DD0014: are the backend processes running? [PASS] DD0008: is the native API responding? [PASS] DD0009: is the vpnkit API responding? [PASS] DD0010: is the Docker API proxy responding? [PASS] DD0006: is the Docker Desktop Service responding? [FAIL] DD0012: is the VM networking working? network checks failed: failed to ping host: exit status 1 [020:15:39:11.336][I] ipc.NewClient: 6410f622-diagnose-network -> \.\pipe\dockerDiagnosticd diagnosticsd [common/pkg/diagkit/gather/diagnose.runIsVMNetworkingOK() [ common/pkg/diagkit/gather/diagnose/network.go:34 +0x107 [common/pkg/diagkit/gather/diagnose.(test).GetResult(0x1161760) [ common/pkg/diagkit/gather/diagnose/test.go:46 +0x43 [common/pkg/diagkit/gather/diagnose.Run.func1(0x1161760) [ common/pkg/diagkit/gather/diagnose/run.go:17 +0x5a [common/pkg/diagkit/gather/diagnose.walkOnce.func1(0x2, 0x1161760) [ common/pkg/diagkit/gather/diagnose/run.go:140 +0x77 [common/pkg/diagkit/gather/diagnose.walkDepthFirst(0x1, 0x1161760, 0xc0004d97d8) [ common/pkg/diagkit/gather/diagnose/run.go:146 +0x36 [common/pkg/diagkit/gather/diagnose.walkDepthFirst(0x0, 0x100000000000000, 0xc0004d97d8) [ common/pkg/diagkit/gather/diagnose/run.go:149 +0x73 [common/pkg/diagkit/gather/diagnose.walkOnce(0xa3d8e0, 0xc00022f918) [ common/pkg/diagkit/gather/diagnose/run.go:135 +0xcc [common/pkg/diagkit/gather/diagnose.Run(0x11617e0, 0xc000626550, {0xc00022fb20, 0x1, 0x1}) [ common/pkg/diagkit/gather/diagnose/run.go:16 +0x1ce [main.checkCmd({0xc0000823d0, 0xc0000823d0, 0x4}, {0x0, 0x0}) [ common/cmd/com.docker.diagnose/main.go:130 +0x105 [main.main() [ common/cmd/com.docker.diagnose/main.go:96 +0x273 [020:15:39:11.336][I] (954e7b90) 6410f622-diagnose-network C->S diagnosticsd POST /check-network-connectivity: {"ips":["10.130.24.148","172.17.224.1","172.25.128.1"]} [020:15:39:11.850][E] (954e7b90) 6410f622-diagnose-network C<-S 2a400e3f-diagnosticsd POST /check-network-connectivity (514.822ms): failed to ping host: exit status 1 [common/pkg/diagkit/gather/diagnose.runIsVMNetworkingOK() [ common/pkg/diagkit/gather/diagnose/network.go:35 +0x182 [common/pkg/diagkit/gather/diagnose.(test).GetResult(0x1161760) [ common/pkg/diagkit/gather/diagnose/test.go:46 +0x43 [common/pkg/diagkit/gather/diagnose.Run.func1(0x1161760) [ common/pkg/diagkit/gather/diagnose/run.go:17 +0x5a [common/pkg/diagkit/gather/diagnose.walkOnce.func1(0x2, 0x1161760) [ common/pkg/diagkit/gather/diagnose/run.go:140 +0x77 [common/pkg/diagkit/gather/diagnose.walkDepthFirst(0x1, 0x1161760, 0xc0004d97d8) [ common/pkg/diagkit/gather/diagnose/run.go:146 +0x36 [common/pkg/diagkit/gather/diagnose.walkDepthFirst(0x0, 0x100000000000000, 0xc0004d97d8) [ common/pkg/diagkit/gather/diagnose/run.go:149 +0x73 [common/pkg/diagkit/gather/diagnose.walkOnce(0xa3d8e0, 0xc00022f918) [ common/pkg/diagkit/gather/diagnose/run.go:135 +0xcc [common/pkg/diagkit/gather/diagnose.Run(0x11617e0, 0xc000626550, {0xc00022fb20, 0x1, 0x1}) [ common/pkg/diagkit/gather/diagnose/run.go:16 +0x1ce [main.checkCmd({0xc0000823d0, 0xc0000823d0, 0x4}, {0x0, 0x0}) [ common/cmd/com.docker.diagnose/main.go:130 +0x105 [main.main() [ common/cmd/com.docker.diagnose/main.go:96 +0x273

[FAIL] DD0032: do Docker networks overlap with host IPs? network bridge has subnet 172.17.0.0/16 which overlaps with host IP 172.17.224.1 [SKIP] DD0030: is the image access management authorized? [WARN] DD0033: does the host have Internet access? unable to fetch http://docker.com/ segment 2022/01/20 15:39:28 ERROR: sending request - Post "https://api.segment.io/v1/batch": context deadline exceeded (Client.Timeout exceeded while awaiting headers) segment 2022/01/20 15:39:28 ERROR: 1 messages dropped because they failed to be sent and the client was closed

Please note the following 1 warning:

1 : The check: does the host have Internet access? Produced the following warning: unable to fetch http://docker.com/

If the host does not have Internet access then containers will also not have Internet access.

The lack of Internet access could be caused by

Please investigate the following 2 issues:

1 : The test: is the VM networking working? Failed with: network checks failed: failed to ping host: exit status 1

VM seems to have a network connectivity issue. Please check your host firewall and anti-virus settings in case they are blocking the VM.

2 : The test: do Docker networks overlap with host IPs? Failed with: network bridge has subnet 172.17.0.0/16 which overlaps with host IP 172.17.224.1

If the subnet used by a Docker network overlaps with an IP used by the host, then containers won't be able to contact the overlapping IP addresses.

Please try configuring the IP address range used by networks: in your docker-compose.yml. See https://docs.docker.com/compose/compose-file/compose-file-v2/#ipv4_address-ipv6_address

Steps to reproduce the behavior

1) Update Docker Desktop for Windows to from 4.3.2 to 4.4.X. I've tested using 4.4.2 and current as of now 4.4.3 2) Attempt a docker pull

dmytro-aleinykov commented 2 years ago

Hi, I have the same issue: it starts failing, once I connect to VPN. It seems like "Proxy settings" are ignored

pirofex commented 2 years ago

I have the same issue. Updated and now my proxy settings seem to be ignored. Only happens if I select linux containers, works with windows containers.

If I downgrade to 4.3.2 everything works as expected.

mhoare1984 commented 2 years ago

I just updated to 4.4.4 and the issue still persists. In the previous check command it mentioned overlapping networks with my Hyper-V default swtich. Since I can't change that address I modified the docker address via the json file as per below

"bip": "10.200.0.1/24", "default-address-pools": [ { "base": "10.200.1.1/24", "size": 28 } ], "fixed-cidr": "10.200.0.1/25",

New check command output after fixing the IP clash where this issue still occurs:

[025:09:07:51.996][I] ipc.NewClient: 9cdf501c-com.docker.diagnose -> \.\pipe\dockerBackendV2 com.docker.service.exe [goroutine 1 [running, locked to thread]: [win/backend/pkg/service.NewClientForPath(...) [ win/backend/pkg/service/service.go:40 [win/backend/pkg/service.NewClient({0x95e8be, 0x13}, {0x0, 0x0, 0x0}) [ win/backend/pkg/service/service.go:29 +0xe8 [common/pkg/diagkit/gather/diagnose.init() [ common/pkg/diagkit/gather/diagnose/ipc_windows.go:23 +0x11f Starting diagnostics

[PASS] DD0027: is there available disk space on the host? [PASS] DD0028: is there available VM disk space? [PASS] DD0031: does the Docker API work? [PASS] DD0004: is the Docker engine running? [PASS] DD0011: are the LinuxKit services running? [PASS] DD0016: is the LinuxKit VM running? [PASS] DD0001: is the application running? [SKIP] DD0018: does the host support virtualization? [PASS] DD0002: does the bootloader have virtualization enabled? [PASS] DD0017: can a VM be started? [PASS] DD0024: is WSL installed? [PASS] DD0021: is the WSL 2 Windows Feature enabled? [PASS] DD0022: is the Virtual Machine Platform Windows Feature enabled? [PASS] DD0025: are WSL distros installed? [PASS] DD0026: is the WSL LxssManager service running? [PASS] DD0029: is the WSL 2 Linux filesystem corrupt? [PASS] DD0015: are the binary symlinks installed? [PASS] DD0003: is the Docker CLI working? [PASS] DD0013: is the $PATH ok? [PASS] DD0005: is the user in the docker-users group? [PASS] DD0007: is the backend responding? [PASS] DD0014: are the backend processes running? [PASS] DD0008: is the native API responding? [PASS] DD0009: is the vpnkit API responding? [PASS] DD0010: is the Docker API proxy responding? [PASS] DD0006: is the Docker Desktop Service responding? [PASS] DD0012: is the VM networking working? [PASS] DD0032: do Docker networks overlap with host IPs? [SKIP] DD0030: is the image access management authorized? [WARN] DD0033: does the host have Internet access? unable to fetch http://docker.com/ segment 2022/01/25 09:08:17 ERROR: sending request - Post "https://api.segment.io/v1/batch": context deadline exceeded (Client.Timeout exceeded while awaiting headers) segment 2022/01/25 09:08:17 ERROR: 1 messages dropped because they failed to be sent and the client was closed

Please note the following 1 warning:

1 : The check: does the host have Internet access? Produced the following warning: unable to fetch http://docker.com/

If the host does not have Internet access then containers will also not have Internet access.

The lack of Internet access could be caused by

No fatal errors detected.

ermalbaj commented 2 years ago

tried with version 4.4.4 and does not work. does not recognize proxy settings. Any workaround for this issue?

punzenbergerpeter commented 2 years ago

I have the same issue. Installed 4.4.4 and my proxy settings seem to be ignored in linux containers.

If I downgrade to 4.3.2 everything works as expected.

fenixcitizen commented 2 years ago

It seems that this exact problem delayed us for couple of days - had to downgrade version to resolve it. I suppose it warrants an emergency patch as soon as possible.

nmofonseca commented 2 years ago

I can confirm the same issue here, 4.4.x doesn't respect proxy settings and with 4.3.2 works fine.

deathchurch commented 2 years ago

I have the same issue as well, shame there is no response to this. I hope its not a feature related to the upcoming licensing changes where proxies would only be supported on Team or Business versions.

PhillipVoyle commented 2 years ago

In the docker engine release notes there is a note about $HTTP_PROXY and $HTTPS_PROXY

IMPORTANT

Due to net/http changes in Go 1.16, HTTP proxies configured through the $HTTP_PROXY environment variable are no longer used for TLS (https://) connections. Make sure you also set an $HTTPS_PROXY environment variable for handling requests to https:// URLs.

Refer to the HTTP/HTTPS proxy section to learn how to configure the Docker Daemon to use a proxy server.

Is this likely related?

dweddig01 commented 2 years ago

I also can confirm this exact issue was resolved for me by downgrading to 4.3.2. When I was troubleshooting the problem, I noticed in Wireshark that there was never any communication to our company's proxy, it would do the DNS lookup than attempt the Client Hello session with one of the IPs returned from registry-1.docker.io. Process Monitor did show the VPNKIT being executed, which is weird, since I am not on a software VPN, but am on a hardware based VPN tunnel using a cisco appliance... though that may normally run regardless.

thaJeztah commented 2 years ago

Thanks everyone for reporting; we're tracking this issue internally and localised the cause; a fix is being worked on.

DarthFotnam commented 2 years ago

Thanks for those that found and have confirmed the issue. I encountered the same and after a full day of diagnostics and rolling back to 4.3.x I decided to come searching to see if anyone else had encountered the same.

I was afraid that Docker had updated the proxy value handling and some of the special characters in our credentials were the cause. These have to be escaped (various ways) on 'Nix platforms, Jenkins build scripts, etc. Please include testing for those use-cases (please, oh, please..)

araisch commented 2 years ago

Please include testing for those use-cases (please, oh, please..)

+1 due to the fact enterprise customers are starting from today (=end of grace period) being the cows producing the milk. And enterprise customers using proxy servers quite often.

bb980701 commented 2 years ago

I have the same issue. Installed 4.4.4 and my proxy settings seem to be ignored in linux containers.

By using the config.json ALL containers receive the HTTP_PROXY,HTTPS_PROXY,http_proxy and https_proxy environment variables see docker help

After installing the latest update 4.4.4, the information about proxies stored in the config.json are not taken into account anymore.

Expected behavior Information stored about proxies inside the /.docker/config.json must be passed as environment variables to all containers according to documentation .

On 4.3.2 , it works only if I force the proxy setting

function dcb(){ docker compose build --build-arg "http_proxy=$HTTP_PROXY" \ --build-arg "https_proxy=$HTTPS_PROXY" \ --build-arg "HTTP_PROXY=$HTTP_PROXY" \ --build-arg "HTTPS_PROXY=$HTTPS_PROXY" }

This is the exact issue as https://github.com/docker/for-win/issues/11938

Please add test-case verifying tha config.json proxies setting are taken into account

ikedam commented 2 years ago

I hit this issue and it's CRITICAL for me as I rely on docker for development. My docker pull never work at all. I tried to downgrade to 4.3.2, but couldn't as the installer rejected to install with "Existing installation is up to date". I believe uninstalling the existing installation results loss of all data.

p1-0tr commented 2 years ago

Thanks for reporting the issue. There is a regression in 4.4.X releases, a fix is on track to be released in 4.5.0. If you'd like to try a dev build which contains the fix, you can find one here:

In case anyone would be looking for a mac build, you can have a look at https://github.com/docker/for-mac/issues/6152.

Sorry for any inconvenience.

tehautanop commented 2 years ago

@p1-0tr Thanks ! We also had this "Client.Timeout exceeded while awaiting headers" problem with 4.4.x versions.

I have just tested the 4.5.0 devbuild in our environment (internal artifactory behind corporate proxy), login is now successful! The only proxy setting I've set is in the Docker Desktop client GUI under "Resources- Proxies."

kmazurek244 commented 2 years ago

Thanks for reporting the issue. There is a regression in 4.4.X releases, a fix is on track to be released in 4.5.0. If you'd like to try a dev build which contains the fix, you can find one here:

* https://desktop-stage.docker.com/win/main/amd64/74266/Docker%20Desktop%20Installer.exe

In case anyone would be looking for a mac build, you can have a look at docker/for-mac#6152.

Sorry for any inconvenience.

Dev build which contains the fix on my laptop shows in docker info the http and https entries. The values defined in UI are there. But no_proxy is not shown.

So it's a partial fix.

DarthFotnam commented 2 years ago

Thanks for reporting the issue. There is a regression in 4.4.X releases, a fix is on track to be released in 4.5.0. If you'd like to try a dev build which contains the fix, you can find one here:

In case anyone would be looking for a mac build, you can have a look at docker/for-mac#6152.

Sorry for any inconvenience.

Apparently that staged build is restricted (I get 404 attempting to access it); and our corporate proxy blocks the link directly to the .EXE. Can it be placed as a ZIP or URL provided to a list/link for the .EXE(s)? This is becoming a real pain as Docker Desktop insists on auto-updating to 4.4.4 every day or so (despite my selections in the UI for checking for upgrades and downloading). I am forced to uninstall and reinstall 4.3.2.

TTimo commented 2 years ago

fwiw, the 4.5.0 dev build linked above brought the behavior back to the same that it was in 4.3.2:

docker pull works again, e.g. docker pull python:3.9 does what it's supposed to do behind a proxy

but FROM python:3.9 in a dockerfile, on a system that never downloaded the image, will fail:

failed to solve with frontend dockerfile.v0: failed to create LLB definition: failed to authorize: rpc error: code = Unknown desc = failed to fetch anonymous token: Get "https://auth.docker.io/token?scope=repository%3Alibrary%2Fpython%3Apull&service=registry.docker.io": dial tcp 54.236.140.233:443: i/o timeout

This can be worked around by pulling first, this happens in 4.3.2 also .. at least the 4.4.4 regression is fixed.

p1-0tr commented 2 years ago

@kmazurek244 - a fix for the no_proxy problem is also lined up for release in 4.5.0. You can find a more recent dev build here - https://desktop-stage.docker.com/win/main/amd64/74506/Docker%20Desktop%20Installer.exe .

DarthFotnam commented 2 years ago

@kmazurek244 - a fix for the no_proxy problem is also lined up for release in 4.5.0. You can find a more recent dev build here - https://desktop-stage.docker.com/win/main/amd64/74506/Docker%20Desktop%20Installer.exe .

Links to the pre-builds are blocked by our proxy (ironically) due to the direct .exe content. And links to the higher resources are blocked on Docker's end (403 permission denied). Can you please provide a navigable link to where this preview EXE can be downloaded without the .exe extension? (ZIP or similar)

kmazurek244 commented 2 years ago

@kmazurek244 - a fix for the no_proxy problem is also lined up for release in 4.5.0. You can find a more recent dev build here - https://desktop-stage.docker.com/win/main/amd64/74506/Docker%20Desktop%20Installer.exe .

Confirmed. Now no_proxy it is passed from UI to docker service. Thanks! :)

p1-0tr commented 2 years ago

@DarthFotnam unfortunately I don't have a good way of doing that, sorry.

p1-0tr commented 2 years ago

A fix for this issue was released in 4.5.0.

ikedam commented 2 years ago

Why is this fix not in the release note? https://docs.docker.com/desktop/windows/release-notes/#docker-desktop-450

docker-robott commented 2 years ago

Closed issues are locked after 30 days of inactivity. This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle locked