Unwanted Proxy

[ricfeatherstone]

Expected behavior

Docker should not run with a proxy settings if there is no proxy configured It should be possible to manually override any proxy settings

Actual behavior

I am not running a proxy on my machine, have no proxy settings, but Docker is running with proxy settings.

docker info | grep -i proxy
HTTP Proxy: docker.for.mac.http.internal:3128
HTTPS Proxy: docker.for.mac.http.internal:3129

Changes in Preferences > Proxies > Manual Proxy Configuration are unable to override this setting

Even manually updating the configuration files fails to change the proxy settings

I've also tried uninstalling, removing ~/.docker, ~/Library/Containers/... and ~/Library/Group Containers/.. folders and reinstalling all without success.

I believe the problem is in the latest update

Information

Docker for Mac: version: 17.12.0-ce-mac47 (72b93a017350990850ddc37cd341bd16fce3e911) macOS: version 10.13.2 (build: 17C88) logs: /tmp/CB9AD7A5-6C90-4D57-B8F4-0B4549E1F6B2/20180119-093345.tar.gz [OK] db.git [OK] vmnetd [OK] dns [OK] driver.amd64-linux [OK] virtualization VT-X [OK] app [OK] moby [OK] system [OK] moby-syslog [OK] kubernetes [OK] env [OK] virtualization kern.hv_support [OK] slirp [OK] osxfs [OK] moby-console [OK] logs [OK] docker-cli [OK] menubar [OK] disk

[bartoszhernas]

Same issue here :/. My microservice cannot connect to external hosts anymore

[bartoszhernas]

The build from here https://github.com/docker/for-mac/issues/2442 fixed the issues for me

[ricfeatherstone]

I think that, or the latest edge might work for me too.

I updated a config file somewhere yesterday, manually setting the proxy trying to override. It didn't work then but it does now. If I can just find that file, remove my changes, I think I'll be good to go

[thaJeztah]

Looks like previously a transparent proxy was used, but this was changed to resolve https://github.com/docker/for-mac/issues/2320, and https://github.com/docker/for-mac/issues/2386

Also may be worth updating to mac49

[ricfeatherstone]

OK, so the build referenced above does not work for me. If I update from that build to the latest Edge, the proxy settings change to what I added yesterday. So I think latest Edge would work if I could find out where it's getting it's config from.

It would be good if switching to manual proxy configuration in the UI and leaving blank would override but it doesn't. Even adding the following to ~/Library/Group\ Containers/group.com.docker/settings.json does not help.

  "proxyHttpMode" : "manual",
  "overrideProxyHttp" : "",
  "overrideProxyHttps" : "",
  "overrideProxyExclude" : "",

Can anyone provide more info on where it's getting it config from?

Somewhere I added the following to a file, which appears to be getting picked up, but I can't find the file

"proxies":
      {
        "default":
        {
          "httpProxy": "docker.for.mac.http.internal:3128",
          "httpsProxy": "docker.for.mac.http.internal:3128",
        }
      }

docker info | grep -i proxy
HTTP Proxy: docker.for.mac.http.internal:3128
HTTPS Proxy: docker.for.mac.http.internal:3128

cat ~/.docker/config.json 
{
  "auths" : {

  }

cat ~/Library/Group\ Containers/group.com.docker/http_proxy.json 
{}

[thaJeztah]

ping @djs55 - who's more familiar with this :innocent:

[ricfeatherstone]

I've deleted everything in

• ~/.docker
• ~/Library/Containers/com.docker..
• ~/Library/Group Containers/group.com.docker..

and reverted to Docker for Mac: version: 17.09.1-ce-mac42 and this has removed the unwanted proxy settings so the issue must be in the update.

I have searched everywhere for the file I added the json configuration for the proxy that was being picked up in the latest edge release but cannot find it.

Where else would docker be picking up config from?

[djs55]

Thanks for your report.

The most recent versions of Docker for Mac have a built-in HTTP and HTTPS proxy (as well as proxies for TCP, UDP, ICMP, NTP and DNS). The docker engine is configured to use these proxies (on docker.for.mac.http.internal) which then either forward to an upstream proxy (if defined) or they fetch the resources themselves. The main benefit is that we avoid having to restart the VM when the proxy settings change.

There were a few bugs in the initial mac47 release, most of which have been resolved in mac49. One bug whose fix has not been released yet is a fix for setting the proxy in the Mac UI to docker.for.mac.host.internal (or docker.for.mac.localhost) and manually running a proxy (or an ssh tunnel to a remote proxy) on the host. This should be fixed in the next update-- if you would like to try it there is a pre-release version here: https://download-stage.docker.com/mac/bysha1/7249e09cc44eb4589e7a339c41cb5096fffdf79d/Docker.dmg

Could you describe a scenario that doesn't work for you with the new proxy scheme?

Thanks!

[ricfeatherstone]

I've just updated as I needed a stable docker environment for things I was working on. The problem is still there.

I am running an OpenShift cluster using oc cluster up and having a proxy configured breaks that, as referenced in the other issue above.

I don't want a proxy setting, I want the previous network settings that worked. Is it possible to add the new proxy as an optional configuration setting, or at least have a simple way to turn it off?

What was the reasoning behind changing the behaviour here, docker on linux does not do this and it was an unexpected (and unwanted) change that appears is unable to be switched off

[djs55]

There are some proxy fixes in the pipeline -- if you'd like to try them then have a look at this comment. Hopefully the fixes will be released in an edge build soon.

If this new build still doesn't fix the problem, could you upload a set of fresh diagnostics? The proxy-related logging in the new build should be better and hopefully will make clear what's going on.

[ricfeatherstone]

Is there any documentation anywhere explaining why a proxy has been introduced?

I would expect to be able to add proxy details if I was using Docker for Mac in a corporate environment where direct access to the internet was restricted. I would not expect to have a proxy where there is no need for one.

As well as breaking for me this change failed the principle of least astonishment and I am trying to understand why this feature was added. As far as I am concerned I have no need for a proxy when using Docker for Mac and do not want one.

Maybe I don't fully understand how networking has been implemented. Was there previously a proxy I was blissfully unaware of that is in fact required in order for Docker for Mac to work?

[kkzz8888]

I am running the latest stable docker for Mac: 18.03.0-ce-mac60. Also having issues with proxy when using openshift. The Preferences Proxies UI won't override with manual proxy (and no proxy):

oc cluster up Using Docker shared volumes for OpenShift volumes Using 127.0.0.1 as the server IP Starting OpenShift using openshift/origin:v3.9.0 ... OpenShift server started.

The server is accessible via web console at: https://127.0.0.1:8443

You are logged in as: User: developer Password:

To login as administrator: oc login -u system:admin

WARNING: An HTTP proxy (docker.for.mac.http.internal:3128) is configured for the Docker daemon, but you did not specify one for cluster up WARNING: An HTTPS proxy (docker.for.mac.http.internal:3129) is configured for the Docker daemon, but you did not specify one for cluster up WARNING: A proxy is configured for Docker, however 172.30.1.1 is not included in its NO_PROXY list. 172.30.1.1 needs to be included in the Docker daemon's NO_PROXY environment variable so pushes to the local OpenShift registry can succeed.

[ricfeatherstone]

I don't understand the reason for introducing the proxy, was this here previously but unnoticed? The issue is there is no way to turn this off and therefore I am stuck on Version 17.09.1-ce-mac42 (21090), Channel: stable, 3176a6af01 and unable to take an upgrade as it breaks what I am working on. The only alternative I can see is to drop Docker for Mac and run Docker natively in a VM, not my favourite option

[evopen]

It's almost July now. Any news? It's mac65 already.

I'm using proxy in Docker for Mac preferences -> proxy. That proxy setting should only be used when pulling images, like the description said. But it also affects proxy inside the container, a transparent proxy.

If I turn that proxy setting off, container will back to normal which use direct connect. Although proxy info still persist in docker info | grep -i proxy. At least this is a workaround. I have to stop using proxy to pull images to gain functionality of container.

[rkgade]

I am running Server Version: 18.03.1-ce and I see HTTP Proxy: docker.for.mac.http.internal:3128 HTTPS Proxy: docker.for.mac.http.internal:3129 thought I haven't configured any proxy.

Here are the details from my workstation.

Raj-MacBook-Pro:group.com.docker raj$ pwd /Users/raj/Library/Group Containers/group.com.docker

Raj-MacBook-Pro:group.com.docker raj$ cat already-enabled-features.json { "releaseNotesV1" : true, "passthroughHTTPProxy" : true, "discoverEEV1" : true } Raj-MacBook-Pro:group.com.docker raj$ cat http_proxy.json {} Raj-MacBook-Pro:group.com.docker raj$ cat settings.json { "hyperkitIpRange" : "192.168.65.0", "proxyHttpMode" : "manual", "diskPath" : "/Users/raj/Library/Containers/com.docker.docker/Data/vms/0/Docker.qcow2", "diskSizeMiB" : 61035, "cpus" : 4, "memoryMiB" : 2048, "displayedWelcomeWhale" : true, "buildNumber" : "24312", "channelID" : "stable", "settingsVersion" : 1, "version" : "18.03.1-ce-mac65", "displayedWelcomeMessage" : true, "linuxDaemonConfigCreationDate" : "2018-07-09 11:40:03 +0000", "dockerAppLaunchPath" : "/Applications/Docker.app" }

Any idea as to why is this happening ?

Also, where is it picking these values from?

[docker-robott]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale comment. Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle stale

[leetrout]

/remove-lifecycle stale

[junior]

/lifecycle frozen

[junior]

Similar to #2715

[junior]

Issue still happening with 2.0.0.0-beta1-mac75.

Containers not getting the proxy envs and ignoring the "exclude"/"no_proxy"

docker run -it alpine env
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=b9ffa7865a33
TERM=xterm
HOME=/root

cat ~/Library/Group\ Containers/group.com.docker/http_proxy.json
{
  "http": "http://www-proxy.us.company.com:80",
  "https": "http://www-proxy.us.company.com:80",
  "exclude": "localhost,127.0.0.1,.us.company.com,.corpcompany.com",
  "transparent_http_ports": [
    80
  ],
  "transparent_https_ports": [
    443
  ]
}

cat ~/Library/Group\ Containers/group.com.docker/settings.json
{
  "proxyHttpMode" : "manual",
  "diskPath" : "/Users/somebody/Library/Containers/com.docker.docker/Data/vms/0/Docker.raw",
  "latestBannerKey" : "DockerCon 2018",
  "diskSizeMiB" : 244140,
  "overrideProxyExclude" : "localhost,127.0.0.1,.us.company.com,.corpcompany.com",
  "defaultMachineMigrationStatus" : 0,
  "memoryMiB" : 65536,
  "overrideProxyHttp" : "http://www-proxy.us.company.com:80",
  "displayedWelcomeWhale" : true,
  "buildNumber" : "27117",
  "cpus" : 10,
  "filesharingDirectories" : [
    "/Users",
    "/Volumes",
    "/private",
    "/tmp"
  ],
  "channelID" : "edge",
  "settingsVersion" : 1,
  "version" : "2.0.0.0-beta1-mac75",
  "displayedWelcomeMessage" : true,
  "overrideProxyHttps" : "http://www-proxy.us.company.com:80",
  "linuxDaemonConfigCreationDate" : "2018-04-27 15:44:30 +0000",
  "dockerAppLaunchPath" : "/Applications/Docker.app"
}

[wind2412]

Also this problem, is there any method to work around with it?

[mapleeit]

What's the process on this one?

Also encountered this problem on:

• Version 2.0.0.3 (31259)
• Channel: stable 8858db33c8

Preferences > Proxies > Manual Proxy Configuration as:

• HTTP: http://web-proxy.oa.com:8080
• HTTPS: http://web-proxy.oa.com:8080

but docker info | grep Proxy says:

HTTP Proxy: gateway.docker.internal:3128
HTTPS Proxy: gateway.docker.internal:3129

So i can't pull image now.

[smekkley]

This issue is being fixed with #2681 and https://github.com/moby/vpnkit/issues/408 Bypassing proxy doesn't work when you try to access tls resources by name in the container.

Workaround is to just add ip addresses or subnets that you want to bypass in the list instead of domains.

@mapleeit docker pull command should work regardless.

[figroc]

Does this proxy affect registry mirrors (#2146)? And the TLS issue (#2681 and moby/vpnkit#408) has not been solved yet .

[defreng]

Unfortunately I'm also affected by this issue...

In my company we need to connect through a proxy for all outgoing HTTPS connections, but the proxy is not being used for HTTPS connection that stay inside the company network. So for example:

• When using docker pull to get the ubuntu image, the proxy must be used
• When I want to use kubectl inside a locally running Docker container it tries to access the HTTPS endpoint of our internal K8s cluster. This fails, because the gateway forwards this request to the proxy.

Is it possible to apply separate proxy settings for the following two things?

• docker pull
• the transparent gateway

[Fire-]

This is still affecting current builds and breaks many corporate development configurations; It's unscalable to add hosts when, for example, some are in AWS or other cloud providers with broad IP ranges.

[jamshid]

Sorry I'm not positive this problem is related but I've been having weird failures when I do a docker-compose pull:

ERROR: for swarm  Get http://192.168.1.50:5000/v2/EXAMPLE/manifests/bin-product-11.2.0: proxyconnect tcp: dial tcp 192.168.65.1:3128: i/o timeout
Get http://192.168.1.50:5000/v2/EXAMPLE/manifests/bin-product-11.2.0: proxyconnect tcp: dial tcp 192.168.65.1:3128: i/o timeout

I finally tracked down this intermittent problem to having this container running, which I normally use for my own Dockerfile builds.

$ docker ps|grep 3128
7c6ca27ae9a2        sameersbn/squid:3.3.8-23                         "/sbin/entrypoint.sh"    37 hours ago        Up 22 hours             0.0.0.0:8123->3128/tcp                                                                                                                                                                       buildenv_httpcache_1

Once I stopped my httpcache container the problem pulling images went away. This is especially weird since the port 3128 is only on the container, it's exposed only as 8123?

Update: maybe the proxy issue is a red herring... I'm asking in https://forums.docker.com/t/is-compose-http-timeout-still-used-when-docker-host-ssh-dockerhost/97344 if docker-compose uses COMPOSE_HTTP_TIMEOUT for DOCKER_HOST=ssh://dockerhost style connections.

[unixfreak0037]

It is 9/11/2020 and this is still an issue. The proxy settings for Docker desktop apply transparently to the containers, even though the current documentation says it does not.

PROXIES Docker Desktop detects HTTP/HTTPS Proxy Settings from macOS and automatically propagates these to Docker. For example, if you set your proxy settings to http://proxy.example.com, Docker uses this proxy when pulling containers.

Your proxy settings, however, will not be propagated into the containers you start.

Any idea when this will get fixed? Anything I can do to help?

[yhaenggi]

This transparent proxy is nuts... Why is there still no way to set a proxy ONLY for pulling/pushing images? I don't need a nasty MITM/transparent proxy that is broken and doesn't get fixed.

[v1r]

I have the same issue in the Docker Desktop for mac v3.0.4. I cannot pull images anymore from the Docker hub..

docker info

OSType: linux
Architecture: x86_64
CPUs: 6
Total Memory: 1.942GiB
Name: docker-desktop
ID: HRKT:4LY6:GVR7:WV6Q:ZY2F:CUBS:CA5K:KXZJ:KD6Z:SN3U:WMPS:FJX2
Docker Root Dir: /var/lib/docker
Debug Mode: true
File Descriptors: 67
Goroutines: 69
System Time: 2021-01-12T02:34:04.477703061Z
EventsListeners: 6
HTTP Proxy: gateway.docker.internal:3128
HTTPS Proxy: gateway.docker.internal:3129
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine

Update Not sure what I just did.. after reading @jamshid comment, I played with docker-compose by typing the following commands:

  516  docker-compose pull hello-world
  517  docker pull hello-world
  518  history

The issue with the docker pull went away.

[jsuchenia]

Looks like starting from version 3.3.0 this proxy has been enabled "by default" and I have a problem with the connection to services on 443 port without proper TLS-with-SNI header

[djs55]

@jsuchenia in 3.3.0 is the transparent HTTPS proxy can now use TLS SNI to match the no_proxy domain names properly. There's a fall-back code path for when SNI isn't available. If it's not working for you could you provide some repro steps?

@smekkley @junior the proxy in 3.3.0 now uses TLS SNI so it should respect domain names in no_proxy settings. Let me know if it works (or not!)

[jsuchenia]

@djs55 Yes, in case of TLS without SNI I'm unable to connect anywhere on port 443 (ticket: https://github.com/docker/for-mac/issues/5568), and I'm unable to turn it off So it could be good to have an option to disable this proxy when we do not need it (like when there are no upstream proxy configured)

[stusmall]

I'm hitting this and I'm also seeing the proxy return incorrect results in some scenarios. If I made an HTTP request to a closed port I will get back an HTTP 500 response back a body "host unreachable"

For some products there is a very significant difference between a open port that responds with an HTTP 500 and a closed port. A great example can be seen below:

➜  ~ docker run --rm -it instrumentisto/nmap 10.0.0.78
Starting Nmap 7.91 ( https://nmap.org ) at 2021-04-13 17:22 UTC
Nmap scan report for 10.0.0.78
Host is up (0.091s latency).
Not shown: 997 closed ports
PORT     STATE SERVICE
80/tcp   open  http
443/tcp  open  https
1443/tcp open  ies-lm

Nmap done: 1 IP address (1 host up) scanned in 1.78 seconds
➜  ~ nmap 10.0.0.78
Starting Nmap 7.91 ( https://nmap.org ) at 2021-04-13 11:22 MDT
Nmap scan report for 10.0.0.78
Host is up (0.078s latency).
Not shown: 999 closed ports
PORT     STATE SERVICE
1443/tcp open  ies-lm

Nmap done: 1 IP address (1 host up) scanned in 20.07 seconds

The first example is portscanning a host from inside an OSX docker image, the second is scanning it from the host itself without the docker proxy in play. These are obviously different in a material way and can have a real impact in the use of the tool.

[unixfreak0037]

This issue has been open since 2018 and it is constantly a thorn in our side, especially for people who are new to our team. We have to enable proxy settings in Docker Desktop when we need to pull images, and then we have to remember to disable proxy settings when we're done because not everything we do uses the proxy. For example, we must not use the proxy when connecting to various internal systems.

It would be fine to have a transparent proxy as an option and even have that set as the default. But we really need an option to be able to turn that off. And you can see from all the responses here that it is an issue for others as well.

Are there any plans to resolve this?

[antonydevanchi]

It happend with me and 3.3.3 now. Fixed by rollback to 3.3.1.

A little bit update wasted time counter: +2 days.

¯\(ツ)/¯

https://docs.docker.com/docker-for-mac/release-notes/

[djs55]

An update on the current state of the HTTP proxy support:

3.3.0 contained a number of proxy bugs but these have hopefully now been fixed in 3.3.4.

Regarding disabling proxies: I have an experimental development build for Mac: https://desktop-stage.docker.com/mac/stable/amd64/64227/Docker.dmg This allows you to enable manual settings but leave them blank like this:

This will unset the http_proxy, https_proxy and no_proxy environment variables used by dockerd.

Independently of that, if you edit ~/Library/Group\ Containers/group.com.docker/settings.json and set

  "vpnKitTransparentProxy": false,

and restart the app, it should disable the transparent proxy.

[antonydevanchi]

@djs55

Seems like vpnKitTransparentProxy has no any effect. But blank proxy settings works fine.

[smekkley]

The workaround I mentioned stopped working as well.

[ascheman]

I really wonder why such a supposed simple problem is still open after more than 3 years? This takes a lot of research and time for just using Docker which has simply worked for a long time without such hassle. I mean, why is it necessary to make things more complex this way? If, for some reason, I would like to get a proxy in place, please let me introduce it.

But why force me to use a proxy? What is the rationale behind it?

[ascheman]

Thanks to @djs55 I can work now with your private build (https://github.com/docker/for-mac/issues/2467#issuecomment-836345057) which seems to be a version 3.4.0.

The question remains why

1. There is no bug fix for more than 3 years?
2. We are forced to use an internal proxy when this makes configuration more complex and error prone?

[lorthirk]

Can we at least expect that the workaround proposed in https://github.com/docker/for-mac/issues/2467#issuecomment-836345057 will find its way in the official 3.4.0 (and possibly 3.3.4 or 3.3.5 if it's too late)?

[luginbash]

Run into this just now, and thanks to the force update feature, I can no longer pull anything. A temporary workaround is to set up a proxy to listen on 3128.

NO, the workaround in https://github.com/docker/for-mac/issues/2467#issuecomment-836345057 doesn't work for me.

[halhenke]

Am using Docker Desktop for Mac v3.3.3 and I get this Error when i enable the "Use new Virtualization Framework".

None of the fixes suggested here helped but disabling this settting got things working again.

[ascheman]

Thanks, @halhenke, I have that enabled too (but never switched it on actively, was not using Big Sur until some days ago). Nevertheless, switching on that feature should then lead to an automatic setup of the respective proxies. And it should be clearly documented what are the consequences.

BTW: I still would understand why the change of the underlying virtualisation should should require the setup of any proxy? In the end the Docker Engine needs to access the upstream network, introducing a proxy into the loop should not have any impact.

[wadefelix]

I bear this bug nealy half a year. Many times the 192.168.65.1:3129 told me a timeout.

[sbarthelmess]

Why was this closed? This is again a bug that:

Docker "host" doesn't honor environmental HTTP_PROXY/HTTPS_PROXY variables, it only does for the containers.

This clearly breaks the use case where someone behind a proxy is attempting to use, say, a "docker pull". And unfortunately, not all of us have ADMIN rights on our work machines, which is of course where you are MOST likely to be behind a proxy. C'mon guys, quit breaking this! As a rule, environmental variables should **trump** all configuration settings.

[mariliah]

Running into this anytime I try to build a docker image as well, will try this to see if it works.

edit: did NOT work

An update on the current state of the HTTP proxy support:

3.3.0 contained a number of proxy bugs but these have hopefully now been fixed in 3.3.4.

Regarding disabling proxies: I have an experimental development build for Mac: https://desktop-stage.docker.com/mac/stable/amd64/64227/Docker.dmg This allows you to enable manual settings but leave them blank like this:

This will unset the http_proxy, https_proxy and no_proxy environment variables used by dockerd.

Independently of that, if you edit ~/Library/Group\ Containers/group.com.docker/settings.json and set

  "vpnKitTransparentProxy": false,

and restart the app, it should disable the transparent proxy.

[cavokz]

I recently encountered this issue. I fiddled with the "Use Virtualization framework", "gRPC FUSE" and "VirtioFS" settings and everything now works again:

Don't know exactly when it got broken, I use Docker daily but I've not been downloading images for a long while.