ports binding in violation of “secure by default”

[richard-delorenzi]

Both ports mode = ingress, and binding to 0.0.0.0, and not giving an error when an IP address is specified but ignored, are in violation of “secure by default”.

There are 3 related issues:

• Using ingress mode by default, is causing all nodes to listen when only one may be expected. Is an inconvenience, and violates security by default.

• Ignoring bind address, and not issuing an error, then binding to 0.0.0.0. Is Is an inconvenience, and violates security by default.

• Not supporting bind addresses, is an inconvenience.

The inconveniences, are making it impossible for me to use swarms. The security problems lead me to expose my data, but I now know how to avoid. However I should not need to think about this, and it could affect other people. (security should be by default).

Usecases

• Use cases that have effected me (definitely real)
  • 1 Bind to port of specific node, because port is in use on other nodes. You can use host mode for this, but the default is a surprise.
  • 2 Bind to port of specific node, because other nodes have public interfaces. You can use host mode for this, but default is in violation of “secure by default”.
  • 3 Bind locally because, you don't want it visible, to other hosts, default is in violation of “secure by default”
  • 4 Bind to 127.0.0.3, because your development machine has a lot of stuff on it, and 127.0.0.1 is in use, by this port. And use /etc/hosts, so that each domain-name is sent to a different container. This works with docker run, but not with compose.
• Other usecases
  • Bind to specific interface e.g. 192.168.0.x, because this is an internal network. default is in violation of “secure by default”
  • Bind to specific specific node, but not constrain the service to run on this node. This is a similar use case to 1 or 2, but with out use of constraints. Traffic would be routed through the swarm.

So in summary

• Ignoring IP address and binding to 0.0.0.0, and default ingress mode, are both in violation of “secure by default”. Errors should be issued if IP-Address specified, and docs updated. Error should be issued if mode not specified (no default), and docs updated.
• Support can then be added for IP addresses in host mode.
• Support for IP addresses in ingress mode, that is limited to local addresses 127.x.x.x, could be added. (different local addresses e.g. 127.0.0.2 and 127.0.0.3 should be treated as different (just passed to OS)).

Allow bind to local address is useful for constrained nodes. Allow bind to particular address, would work for constrained node, or routed through the swarm to one of the addresses on one of the nodes (may be ingress mode only). This routing is done already

Steps to reproduce the issue:

1. Create a compose file with a ports section. Specify an ip address.
2. Deploy into a swarm

Describe the results you received: The exposed port was on all nodes (public and private), and on all interfaces (public and private). Data was exposed.

Describe the results you expected: Port published on one node, on IP 127.0.0.3

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker version:

Client:
 Version:       18.03.0-ce
 API version:   1.37
 Go version:    go1.9.4
 Git commit:    0520e24
 Built: Wed Mar 21 23:11:46 2018
 OS/Arch:       linux/amd64
 Experimental:  false
 Orchestrator:  swarm

Server:
 Engine:
  Version:      18.03.0-ce
  API version:  1.37 (minimum version 1.12)
  Go version:   go1.9.4
  Git commit:   0520e24
  Built:        Wed Mar 21 23:10:17 2018
  OS/Arch:      linux/amd64
  Experimental: false

Output of docker info:

Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 10
Server Version: 18.03.0-ce
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 11
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: active
 NodeID: l3z84lb1ry6hjae5gba5z2n21
 Is Manager: true
 ClusterID: plfkxxw39gi4jgocsb8f2u368
 Managers: 1
 Nodes: 2
 Orchestration:
  Task History Retention Limit: 5
 Raft:
  Snapshot Interval: 10000
  Number of Old Snapshots to Retain: 0
  Heartbeat Tick: 1
  Election Tick: 3
 Dispatcher:
  Heartbeat Period: 5 seconds
 CA Configuration:
  Expiry Duration: 3 months
  Force Rotate: 0
 Autolock Managers: false
 Root Rotation In Progress: false
 Node Address: 192.168.1.253
 Manager Addresses:
  192.168.1.253:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: cfd04396dc68220d1cecbe686a6cc3aa5ce3667c
runc version: 4fc53a81fb7c994640722ac585fa9ca548971871
init version: 949e6fa
Kernel Version: 3.16.0-5-amd64
Operating System: Debian GNU/Linux 8 (jessie)
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 7.737GiB
Name: delorenzi
ID: UGLS:QVO6:VYVK:RKOT:2WVD:52KF:JHUF:2WDL:LGON:4H4I:U2AM:D2CY
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Username: mrdelorenzi
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

WARNING: No memory limit support
WARNING: No swap limit support
WARNING: No kernel memory limit support
WARNING: No oom kill disable support
WARNING: No cpu cfs quota support
WARNING: No cpu cfs period support

Additional environment details (AWS, VirtualBox, physical, etc.):

Physical Debian Gnu/Linux 8.10

Linux xxxxxxxxx 3.16.0-5-amd64 #1 SMP Debian 3.16.51-3+deb8u1 (2018-01-08) x86_64 GNU/Linux

[bradmunz79]

Is there any rational reason why swarm prevents you from picking the IP address on the node that you'd like to bind to? What if you want to deploy multiple applications on a single node, each with it's own IP address? It seems like a completely reasonable use case.

[cpuguy83]

If you want to pick the node ip, you must use host mode routing which is an option in the port binding.

[richard-delorenzi]

@cpuguy83 I just read about host mode https://docs.docker.com/network/host/ it does not seem to have anything that would help with @bradmunz79 's case. Even if the software in the container, can choose to bind to only one address (not mentioned in this intro), then it relies no the containerised software to do the correct thing. However this is the containers job. We should be able to tell a container to map an internal port to an external host port e.g. 127.0.1.1

[richard-delorenzi]

Are we at least warning the user, if we try to do this? “Secure by default”, tells us that we should error, and refuse to start.

[cpuguy83]

No, it does not rely on the application to do the right thing, it's a config in the swarm service spec.

Why would a swarm service bind to localhost? Why not just run a normal container at that point?

[thaJeztah]

The default is "no ports are published", which I don't think violates "secure by default". For services that are not meant to be accessible from outside the host, you can "not" publish ports, and connect them to an internal network for other services or containers to connect to them.

Being able to connect to a specific interface is still an open feature-request (https://github.com/moby/moby/issues/24317), but non-trivial (and related: ip-addresses for services https://github.com/moby/moby/issues/24170).

(MacVlan networking may be an option for some use-cases https://docs.docker.com/network/macvlan/, http://killcity.io/2017/08/12/setup-docker-swarm-with-macvlan-plus-consul-and-autoscaling.html)

As to

Errors should be issued if IP-Address specified, and docs updated. Error should be issued if mode not specified (no default), and docs updated.

A warning was added when deploying through a stack (https://github.com/docker/cli/pull/1017), and an error will be produced when attempting to bind to an IP-address;

docker service create --publish 127.0.0.1:8080:80 nginx:alpine
invalid argument "127.0.0.1:8080:80" for "-p, --publish" flag: hostip is not supported