run weave with fewer privileges

[rade]

9 covers running with in a non-privileged container. Which is quite a challenge.

As a hopefully easier step, we could try running with --cap-add=NET_ADMIN instead of --privileged. The former was only introduced in Docker 1.2.0, so we'd have to give up compatibility with earlier versions. I seem to remember that weavedns or perhaps weavetools requires a more recent version anyway, but I cannot find anything to that effect in our docs, so perhaps I am imagining. Perhaps @squaremo knows.

NB: This issue is about the weave container, though we should also look at weavedns. PS: Does talking to the docker daemon, as we do in weavedns, require extra privileges? If so weave may well require the same post #22.

[rade]

I have just run a quick experiment, and it looks like running with --cap-add=NET_ADMIN is sufficient for the router, when using either sleeve or fastdp. The router starts fine and talks to other peers.

I haven't looked at the proxy or plugin.

[rade]

I have successfully run our entire test suite with this patch applied

diff --git a/weave b/weave
index 71d6f18..1774bb0 100755
--- a/weave
+++ b/weave
@@ -1590,7 +1590,7 @@ launch_router() {
     # Set WEAVE_DOCKER_ARGS in the environment in order to supply
     # additional parameters, such as resource limits, to docker
     # when launching the weave container.
-    ROUTER_CONTAINER=$(docker run --privileged --net=host -d --name=$CONTAINER_NAME \
+    ROUTER_CONTAINER=$(docker run --cap-add=NET_ADMIN --net=host -d --name=$CONTAINER_NAME \
         $(docker_sock_options) \
         -e WEAVE_PASSWORD \
         -e WEAVE_CIDR=none \

@dpw points out that things may not work so well on SELinux. The docker run docs on security are somewhat cryptic.

[rade]

Bad news. The proxy fails to start even with --cap-add=ALL, exploding in netns.GetFromPath.

[rade]

The plugin appears to work fine with --cap-add=NET_ADMIN.

[dpw]

Bad news. The proxy fails to start even with --cap-add=ALL, exploding in netns.GetFromPath.

AppArmor, maybe? Search for apparmor="DENIED" in /var/log/syslog.

[rade]

AppArmor, maybe

Indeed!

Feb 26 15:16:19 xps kernel: [627265.635303] audit: type=1400 audit(1456499779.462:91): apparmor="DENIED" operation="ptrace" profile="docker-default" pid=13803 comm="weaveproxy" requested_mask="read" denied_mask="read" peer="unconfined"

[rade]

so, ptrace just means "access /proc", I think, which is of course what netns.GetFromPath does.

The following makes the proxy work w/o --privileged:

--security-opt=apparmor:unconfined --cap-add=SYS_ADMIN --cap-add=SYS_PTRACE

That's on my desktop. in the test VM I need --security-opt=seccomp:unconfined ....

[rade]

Related: Automatically generate seccomp profiles.

[dpw]

--cap-add=SYS_ADMIN

That's a shame: https://lwn.net/Articles/486306/

[rade]

The following makes the proxy work w/o --privileged

...but more is needed to actually be able to start containers through it. I just had to add NET_ADMIN to get around

WARN: 2016/02/26 17:12:42.175636 Attaching container f316e79c169c2f5a94fce7241540bf40660becd711e6dd1f689e6f08ecae8e0d to weave network failed: Cannot set device feature settings: Operation not permitted

and now I am faced with

WARN: 2016/02/26 17:19:10.425399 Attaching container 89409041a2b4ea112bf575b8c48de1043dfa65967125797ca3c5e714a9e0f856 to weave network failed: sh: can't create /proc/sys/net/ipv4/neigh/weave/base_reachable_time: Read-only file system

[rade]

btw, it's curious that we do not need NET_RAW, given #9 and our use of pcap.

[dpw]

btw, it's curious that we do not need NET_RAW, given #9 and our use of pcap.

Seems that docker grants CAP_NET_RAW by default:

# docker run --rm ubuntu capsh --print
Current: = [...],cap_net_raw,[...]

[bboreham]

Cross-referencing https://github.com/docker-library/official-images/issues/1528