Access the Containers from Host like on Linux

[KartoffelToby]

This is a kind of a requst

Hello there,

i have a testing development scenario build with docker containers. on linux machines i can access them via the conatinaer IP (172.17.0.X) and intacting via the exposed Port.

But on Docker for Mac this isn't possible because i dont know the IP from the VM

with the Toolbox (Docker Machine) i can route 172.17.0.x to docker machine ip.. is there any way to do that with Docker for Mac?

I need this because i have multiple Database Containers each with the same Port... (so -p istn't the answer ;))

[AbimbolaE]

My current solution is to create the containers using Docker Machine (A linux VM which is available under another IP address) and route all the traffic of the containers to the docker machine VM.

sudo route -n add -net 172.18.0.0/16 192.168.99.100

You can get the network range of your docker containers using docker inspect <container_name> and the IP address of your docker machine VM using docker-machine ip <machine_name>

[burdiyan]

The @AbimbolaE solution seems to be only working with VirtualBox driver, well at least not with hyper kit nor xhyve. Does anyone have any idea why?

[AlmirKadric]

@AbimbolaE your solution is similar to what I created in my package, except it doesn't require the use of docker machine. Take a look here https://github.com/AlmirKadric-Published/docker-tuntap-osx

[burdiyan]

I ended up running an OpenVPN server in a container, forwarding it's port and connecting to it my host's OpenVPN client. That was the easiest way for me to make it feel like my mac is in the same network as docker and Kubernetes cluster running in Docker for Mac Edge.

Can maintainers explain why there is no way to bridge VM's network with host's network out of the box? What are the implications? It seems like Hyperkit allows that (https://github.com/moby/hyperkit/issues/178#issuecomment-357240684), so there should be a way.

It's just so frustrating having to constantly forward ports when developing locally, and would be so great just to address containers and kubernetes pods by IP and names from anywhere in your host OS.

Maybe embedding some sort of VPN solution would work? https://www.tinc-vpn.org for example?

[AlmirKadric]

@burdiyan i can't tell you exactly what the stance is right now with docker maintainers. But I can tell you the experience of the feedback I have gotten from them for the longest time (more than a year, I've been looking at this for quite some time). I put this here in case a docker-for-mac maintainer doesn't actually respond to share my experience. If there is anything wrong with my information please correct me.

From what I have seen, this issue is not a technical one but more a philosophical one. The docker philosophy is to keep environments contained and isolated. However during development (especially on multiple projects at the same time) this presents some issues about local vs containerized etc. Secondly docker doesn't want to introduce any external dependencies to have their software work (e.g. tuntap). Instead they want to make sure that the installation of docker provides the features it needs to. Lastly (and probably more speculation), the docker team right now is more focused on production benefits, so these kinds of quirky (perhaps a misplaced negative connotation) developer requirements aren't as high priority to them, so they are just sweeping it under the rug for now from what I have seen.

Unfortunately docker-for-mac isn't open source (I haven't been able to find it, if it is please let me know the url and I will gladly try and contribute a fix for this) so it doesn't seem a community fix is possible at this stage. In the year and a bit I've been trying different ways to push this I have received a passive response from the docker team at best. I really do hope this becomes a priority at some point and they fix the issue, especially since its not an issue on windows or linux! If you check previous posts, you will see the progress of this discussion on many different forums. I've tried to aggregate the information on several occasions.

The only hope I can see at this time is the experimental network driver that is being developed, but its progress is slow and...well...experimental (I'll be looking at this at some point to update my above mentioned package so that tuntap isn't a requirement anymore).

[onpaws]

Not a Docker expert, just wanted to share a something I figured out today that was super helpful. With docker-machine it's possible to expose Docker containers to localhost on a port by port basis via SSH.

Just setup an SSH port forward for each container's ports. docker-machine ssh default -L 80:localhost:80 -L 123:localhost:123 ...

Now you can, from your Mac, point to e.g. http://localhost/ and browse your container.

Apologies if it's obvious to some of you guys or already been shared. I'm still new to Docker and despite reading lots of docs, threads, and GH issues this wasn't obvious.

Today this technique allowed me to use 3rd party containers configured to point to localhost, on a Mac, with the tools I already had installed. Thought it was an elegant way to work things out, and really saved my day.

Looking forward to using more Docker in the future!

[AlmirKadric]

@onpaws are you familiar with port forwarding within docker? essentially you can do just that without the need for ssh tunnels. You can do this using the -p flag when running your docker command or using the ports option from within your docker-compose.yml file. Is there any specific reason you would want to use a SSH tunnel instead?

The reason we don't opt for that within this thread is if you have services which need to use the same port it would clash (e.g. two mysql containers listening on 3306). Instead its easier to just go to the port on an ip address which is mapped to the container.

[onpaws]

Thanks Almir! What you’re saying makes sense. The docker-compose.yml file I started with has ports options, but for whatever reason browsing the containers via localhost, didn’t work, only via IP which the software on the container didn’t like.

Welcome to try it if you like, the file is here: https://github.com/fbsamples/f8app/blob/master/docker-compose.yml

[tcldr]

Just throwing my thoughts in to the mix as I think this applies in my own situation.

I'm using docker to provide my local development environments. Currently, remembering and assigning ports for the various environments requires a bit of juggling and is a bit of a pain. What I'd like to be able to do is have a simple mechanism for assigning a .test domain name to a particular project.

It would be great if Docker for Mac facilitated this. Obviously it would need to be done in such a way that doesn't interfere with existing features.

One way I imagine this could work user experience wise:

1. User creates docker compose project. As usual, a network is assigned – either my_app_default or another user-defined name.
2. A new docker specific 'hosts' file (or ideally, a preferences panel within the Docker for Mac app) allows for the mapping of project name to domain name. i.e *.my_app_default.test points to my_app_default.
3. User can now access containers via domain name. i.e. www.my_app_default.test or db.my_app_default.test

I realise from the above that this probably isn't simple, but even if it requires running an additional container with a dns server and rerouting the host dns queries – it would seem a worthwhile price.

[ramarnat]

@ericdwhite Using the latest version, this experimental feature does not seem to work, after changing the database state to both neither the bridge100 nor the en1 shows up. I have a docker0 and eth0.

I also see that the host does not have iptables

Where can I go look to see if the experimental bits are still around?

edit: I also tried it with the edge version:

Client:
 Version:      18.06.0-ce-rc3
 API version:  1.38
 Go version:   go1.10.3
 Git commit:   cbfa3d9
 Built:        Thu Jul 12 05:15:46 2018
 OS/Arch:      darwin/amd64
 Experimental: false

Server:
 Engine:
  Version:      18.06.0-ce-rc3
  API version:  1.38 (minimum version 1.12)
  Go version:   go1.10.3
  Git commit:   cbfa3d9
  Built:        Thu Jul 12 05:26:13 2018
  OS/Arch:      linux/amd64
  Experimental: true

[AlmirKadric]

@ramarnat not a 100% answer, but in my personal project which i use to work around this issue, I keep a list of commands such as iptables access (command changed a while ago). You can see the commands I run here: https://github.com/AlmirKadric-Published/docker-tuntap-osx/issues/11

IP tables specifically is now docker run --rm --privileged --pid=host debian nsenter -t 1 -m -u -n -i iptables-save

hope that helps

[ramarnat]

@AlmirKadric Thank you, I will use your tuntap work around instead of the experimental stuff.

[qoomon]

I've created a docker container to access host ports => https://github.com/qoomon/docker-host

[docker-robott]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale comment. Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle stale

[AlmirKadric]

/remove-lifecycle stale

[diegolaciar]

+1 please add this feature to docker for mac

[paulomanrique]

2 years and nothing yet? Seriously, have anyone tried to work with microservices without this?

[dimaspivak]

@paulomanrique If I were you, I'd pursue a refund of the money you've paid for Docker for Mac. ;)

[erszcz]

@paulomanrique

2 years and nothing yet? Seriously, have anyone tried to work with microservices without this?

Yes. Run your service in a container attached to the Docker network that your service environment is deployed in and the problem is gone. Docker Compose is your friend for environment setup.

One handy trick is not to start the service under development via Compose - just the services it requires - and run it manually with:

docker run -v $PWD:$PWD -w $PWD --network compose_env_network ...

Edit natively on Mac, build, test, generate packages, whatnot inside the container. Thanks to the mounted volume all the paths in build artifacts are consistent with your project checkout on Mac. Thanks to the --network flag all the services are accessible.

[paulomanrique]

@dimaspivak Thanks for enlightening us.

[artemkaint]

@dimaspivak it would be nice if docker had premium version with network support. But in my opinion docker-on-mac is a dying product.

Really all available methods to get bi-direction connection between docker and host are crutches. And if situation has not been changing for more than 2 years we unfortunately need to search approach for development on mac. Existing solutions are not really suitable for big products

[scottlaplante]

After several attempts to coerce Docker For Mac to be the product we wanted it to be out of the gate, we stumbled upon Traefik.

It does what we want, how we want, with effectively zero downside. It is, in short, a game changer.

I wish everyone luck and success.

https://traefik.io

[artemkaint]

Thank you for advice. But reverse proxy is not the same thing that everyone here needs

[njwest]

Yeah docker isn't an out-of-the-box reverse proxy, it is a containerization solution... but cheers!

+1 on this feature, although it looks like it requires new xhyve features

[zoltan-nz]

A possible solution: setup dnsmasq with loopback alias

Articles:

• Using dnsmasq and loopback alias to solve Docker for Mac networking issue
• Use dnsmasq instead of etc hosts
• Persistent loopback on macOS

1. Setup loopback alias

Temporary loopback alias

$ sudo ifconfig lo0 alias 10.254.254.254

Permanent loopback alias

$ cat << EOF | sudo tee -a /Library/LaunchDaemons/my.awesome.loopback.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>Label</key>
    <string>my.awesome.loopback</string>
    <key>ProgramArguments</key>
    <array>
        <string>/sbin/ifconfig</string>
        <string>lo0</string>
        <string>alias</string>
        <string>10.254.254.254</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
  </dict>
</plist>
EOF

2. Setup dnsmasq

Install dnsmasq with brew. dnsmasq website: http://www.thekelleys.org.uk/dnsmasq/doc.html

brew install dnsmasq

Add your own personal config file to the default config file:

# Create an `/etc` folder in your brew installation folder if not exists
$ mkdir $(brew --prefix)/etc
$ cat << EOF | tee $(brew --prefix)/etc/dnsmasq.conf
conf-file=/Users/yourname/.dnsmasq/dnsmasq.conf
EOF

Create your personal config file in ~/.dnsmasq/dnsmasq.conf

(Don't use .dev, Chrome automatically redirect it to https://something.dev.)

cat << EOF | tee ~/.dnsmasq/dnsmasq.conf
server=8.8.8.8
server=8.8.4.4
address=/.local/10.254.254.254
address=/.test/10.254.254.254
address=/.loc/10.254.254.254
EOF

(You can use tee -a option to append content to the file instead of overwriting it.)

The full sample config file: https://github.com/imp/dnsmasq/blob/master/dnsmasq.conf.example

Restart dnsmasq

sudo brew services restart dnsmasq

3. Change your main network DNS to 10.254.254.254 in your macOS System Preferences -> Network -> Advanced -> DNS

[danquah]

@zoltan-nz It's a really nice thing to have in place - I personally use Dory ( https://github.com/FreedomBen/dory ) for that. It's not routing though (where you can access the individual containers by their own ip) so it's not the solution this issue is looking for.

[genevera]

Routing to the container IPs is pretty simple. I use corectl and with a little bit of route work I access everything directly. The same can likely be done with docker for mac.

sudo route -nv add <ip block used internally by dockerd> <ip address of vm running docker>
sudo ifconfig <bridge device used by VM> -hostfilter <enX - iface attached to bridge device linking VM to outside world>

eg

sudo route -nv add 172.21.0.0/24 192.168.64.46
sudo ifconfig bridge111 -hostfilter en8
➜ docker inspect --format="{{.NetworkSettings.IPAddress}}" elasticsearch-1
172.21.0.3
➜ curl 172.21.0.3:9200 -v
*   Trying 172.21.0.3...
* TCP_NODELAY set
* Connected to 172.21.0.3 (172.21.0.3) port 9200 (#0)
> GET / HTTP/1.1
> Host: 172.21.0.3:9200
> User-Agent: curl/7.64.1-DEV
> Accept: */*
>
< HTTP/1.1 200 OK
< content-type: application/json; charset=UTF-8
< content-length: 497
<
{
  "name" : "syslog-elasticsearch-1",
  "cluster_name" : "syslog",
  "cluster_uuid" : "W7AoAq-0Rp6MKxdzqq8Tpw",
  "version" : {
    "number" : "6.7.1",
    "build_flavor" : "oss",
    "build_type" : "tar",
    "build_hash" : "2f32220",
    "build_date" : "2019-04-02T15:59:27.961366Z",
    "build_snapshot" : false,
    "lucene_version" : "7.7.0",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}
* Connection #0 to host 172.21.0.3 left intact
* Closing connection 0

Add in some sort of svc discovery (like via docker-gen) and you can hit everything via hostname.

[PMacDiggity]

Only a partial solution as it doesn't' give you the full info (such as the requiter's real IP & hostname etc.)

[genevera]

I'm sorry, could you please elaborate?

[PMacDiggity]

Any incoming connections will appear to be coming form the NAT, so if you are for exampling running a proxy or a web server, and you want to know the IP of the original client, you won't get that, all you will see in the logs is 172.21.0.1, so you can't make rules, or use a WAF etc.

[genevera]

Any incoming connections will appear to be coming form the NAT, so if you are for exampling running a proxy or a web server, and you want to know the IP of the original client, you won't get that, all you will see in the logs is 172.21.0.1, so you can't make rules, or use a WAF etc.

OP didn't ask for that..?

Also you don't see 172.21.0.1. You will see the IP of the host making the request:

[qoomon]

my docker image to access docker host ports now supports tcp and udp connections https://github.com/qoomon/docker-host

[pauldraper]

I'm curious: what prevents you from running multiple database containers and mapping each to a different port and then using a complete sockaddr (address + port rather than address alone) to refer to each?

That approach requires I decide ahead of time what ports are accessible, including adding any debug ports.

And then some programs like passive FTP servers require an entire range of ports be open.

It's simplest to simple allocate an IP address for the guest machine (an IP address for a machine is after all, what IP address are for; this whole ad-hoc developer-implemented NAT-via-localhost is weird).

Presumably, whoever created Docker for Windows would agree with me since they created the bridged network hvint0.

If after three years it is too difficult to have a native Docker for Mac solution, I would at least recommend pointing folks to https://github.com/AlmirKadric-Published/docker-tuntap-osx which solves this problem with TunTap and a handful of shell scripting.

[pauldraper]

I would also add: as part of Docker's obsession with userspace networking, last I checked it launches a separate process for every single port, making my situation even less tenable.

So if you have an FTP server with 200 passive ports, you are now constantly running 200 processes.

[gustavocoding]

+1

[fardjad]

I've taken a different approach to work around this limitation for my own use-cases. Docker Network Exposer runs an OpenVPN server that you can connect to from the host machine. It allows you to reach other containers on the same Docker network. It also pushes a route to the OpenVPN client, so you don't have to install or remove the routes on the host machine manually. The image also generates an additional hosts file that can be used by Dnsmasq to resolve Docker container names on the host machine.

[lucasmenezesaurea]

nothing yet?

[qoomon]

Although sad, I think my docker container is still the best solution https://github.com/qoomon/docker-host

[burdiyan]

Slack just open sourced their global overlay network solution https://slack.engineering/introducing-nebula-the-open-source-global-overlay-network-from-slack-884110a5579

It’s a mix of IPSec and TincVPN, but simpler, faster and written in Go. This may be useful for creating an overlay network between the host machine and containers running within Docker VM.

[burdiyan]

After not using minikube for a long time, I checked it out again. And it turns out that now there's minikube tunnel command that does exactly what I wanted for years! And it even works with the hyperkit driver too.

So now I wonder, why Docker for Mac doesn't expose the hyperkit VM IP address on the Mac? Is there any reason for that? Why minikube can do that using hyperkit?

Apparently all that minikube tunnel is doing is adding couple routes and using the VM as a gateway, which is something Docker for Mac could be doing too, if only we had access to the VM ip.

It's a bit sad, but I'm thinking to switch over to minikube from Docker for Mac, and the trick is that you can setup minikube with the same CIDR for Kubernetes services as the Docker CIDR, and this way minikube tunnel allows you to interact with plain docker containers by their IP.

Are there any plans to follow up on this for Docker for Mac?

[docker-robott]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale comment. Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle stale

[Mahoney]

/remove-lifecycle stale

[jasmas]

The Apple M1 Tech Preview uses the new Virtualization.framework in Big Sur which resolves this issue because a bridge interface is now connected between the host and container VM using the virtio driver and a NAT is done on the host as well. Would it be possible to get the Developer Preview released for Apple M1 compiled and released on Intel as well? It would be useful so we can test. https://github.com/docker/roadmap/issues/142

[Mahoney]

For anyone else still chasing this - I packaged up @AlmirKadric 's solution as a brew formula: brew install mahoney/tap/docker-tuntap-osx

Unfortunately there are a couple of manual sudo steps to do as well, listed in the caveats when you do the install.

Works using https://github.com/Mahoney/docker-lifecycle-listener

[Dmitry-N-Medvedev]

For anyone else still chasing this - I packaged up @AlmirKadric 's solution as a brew formula: brew install mahoney/tap/docker-tuntap-osx

Unfortunately there are a couple of manual sudo steps to do as well, listed in the caveats when you do the install.

Works using https://github.com/Mahoney/docker-lifecycle-listener

FYI:

Warning: Calling depends_on :tuntap is deprecated! There is no replacement.
Please report this issue to the mahoney/tap tap (not Homebrew/brew or Homebrew/core):
  /usr/local/Homebrew/Library/Taps/mahoney/homebrew-tap/Formula/docker-tuntap-osx.rb:10

[Mahoney]

Yes, that's unavoidable. If you want to discuss it further I'd suggest doing so here: https://github.com/Mahoney/homebrew-tap/issues

[pauldraper]

The Authoritative Guide to Why Docker Choosing Layer 4 Proxy Sucks:

1. It spawns a new process for every port. An FTP server usually needs a sizeable number of ports for data transfer. Making 100 ports available requires 100 processes.
2. It only works for TCP and UDP, excluding common protocols such as ICMP, AH, and ESP.
3. It loses the source IP address, preventing observability and debugging.
4. It ties up a port on the host, making it more difficult to run multiple instances of a service.
5. All these things are regressions compared to VMs.
6. Subjective, but it's really just weird. Docker should not care at all what transport protocols are being used.

[Mahoney]

7. You can't assign a name to a port. You can assign a name to an IP address. Humans are much better at remembering names than numbers. Services have default ports, saving you having to remember any number at all.

[pauldraper]

The Apple M1 Tech Preview uses the new Virtualization.framework in Big Sur which resolves this issue because a bridge interface is now connected between the host and container VM using the virtio driver and a NAT is done on the host as well.

Does this work?

[blakebarnett]

The Apple M1 Tech Preview uses the new Virtualization.framework in Big Sur which resolves this issue because a bridge interface is now connected between the host and container VM using the virtio driver and a NAT is done on the host as well.

Does this work?

I verified this works

1. Enable the experimental support in docker-for-mac:

2. Find the bridge interface it creates (bridge100 for me), create a route for it, something like:

sudo route -v add -net 172.18.0.1 -netmask 255.255.0.0 192.168.64.2