search

weaveworks / weave

Simple, resilient multi-host containers networking and more.
https://www.weave.works
Apache License 2.0
6.62k stars 671 forks source link

populate DNS from CNI #2404

Open philwinder opened 8 years ago

philwinder commented 8 years ago

The context of this issue involves Mesos, but the same reasoning should apply to k8s.

When creating a unified Mesos container (Docker or simple process) with CNI support, Mesos will submit an add request to the Weave CNI plugin. The plugin then correctly assigns an IP address, and that address is added to the weave network.

However, weave DNS and weave ps does not know about this container, because it is not a process and/or a docker container. It is a self contained, isolated namespace and therefore not visible to the external process. But this should not matter. Since we receive state updates from the CNI request (i.e. add and del) we should be able to maintain the weave state manually.

Hence, I request that this functionality is added to the weave cni plugin.

Pseudocode

receive and parse NetConf
create and add ip address to network
assign to container
(*) populate weave DNS with container name and ip
(*) populate any internal weave state so that `weave ps` correctly shows mesos containers.
(*) return dns info in result packet (although probably not used by Mesos)

(*) = new.

DNS usage

Then for Mesos (and probably k8s) we can edit /etc/resolve.conf to use the local weave DNS server as the nameserver and everyone can just refer to task-name.weave.local (or whatever comes through as the name in the CNI packet - visible in weave ps)

Caveats

If the CNI source also uses the standard user-space docker daemon, weave dns may pick up the container from the standard docker plugin. We could disable the docker plugin in this case, or prefix the received container names with cni- or put them on a different domain *.weave-cni.local or something.

rade commented 8 years ago

How would CNI get hold of the container name? I don't think it's part of the 'add' request.

rade commented 8 years ago

(*) populate any internal weave state so that weave ps correctly shows mesos containers.

Please raise a separate issue for that. It is completely orthogonal to the DNS issue.

bboreham commented 8 years ago

How would CNI get hold of the container name? I don't think it's part of the 'add' request.

Container ID is, and from there one can get to anything else via the Docker API

rade commented 8 years ago

Container ID is, and from there one can get to anything else via the Docker API

Can one? ...

it is not a process and/or a docker container. It is a self contained, isolated namespace and therefore not visible to the external process.

bboreham commented 8 years ago

OK, then it would be up to Mesos to pass in the necessary information as 'extra arguments' within the CNI spec.

rade commented 8 years ago

it would be up to Mesos to pass in the necessary information as 'extra arguments' within the CNI spec

@philwinder does it?

rade commented 8 years ago

@bboreham is this a dup of #2056?

bboreham commented 8 years ago

is this a dup of #2056?

The 'DNS usage' part of this issue is in the same territory. #2056 notes that K8s does not (yet) respect the DNS info; this issue suggests that we unilaterally override what Mesos puts in place. It would be better to have an agreed handshake with Mesos.