Placeholder for per-container IP address support

[SEJeff]

There is a feature in the just released Mesos 0.25 for assigning up addresses to each container.

This placeholder is to add a way of specifying a static ip for each broker or the beginning of functionality to make this framework play well with this fantastic new feature in mesos.

[joestein]

Setting the bind for the broker (which is what is advertised from Kafka) is an existing feature on broker add/update commands.

--bind-address        broker bind address (broker0, 192.168.50.*, if:eth1). Default - auto

glad 0.25 is out now with more better network support per container, we have been going by https://github.com/mesosphere/net-modules as reference for future implementations we can keep this issue open as we solidify what (if anything) the scheduler needs for 0.25 support and use for virtual network providers too.

[JohnOmernik]

So I see the IP Per container as a great to facilitate a "self services" type arrangement with kafka-mesos across nodes. I.e. you could have multiple clusters of kafka mesos running easily. (Actually, I am curious if someone who knows the innards better on this would be willing to speak on the subject) basically, I am thinking that a cluster could have some "production" level kafka clusters, managed at the production level, but then also allow users to create "development" kafka clusters for their own development and playing. Using APIs, this could be pretty straightforward and using roles, could limit it so dev users "clusters" don't affect the production cluster or clusters. Obviously IP per container makes things like this much easier, but are there any other tweaks we'd have to make to do something like this with Kafka Mesos?

[SEJeff]

@joestein I'm not sure that is exactly what I meant. I'm looking to be able to say: "Broker id1 gets ip 1.2.3.4", and have magic happen + the framework bring up broker 1 that then has (via mesos network isolation magic in 0.25) an ip of 1.2.3.4.

Not honestly sure how this would work, but it seemed worth the placeholder for adding this support once it is a bit more fleshed out in mesos proper. I'm using this framework to build "kafka as a service" internally for different teams. That would be clutch for some less technical teams who just want to hardcode configs into their C++ apps instead of hitting the api to get the host/port combinations.

[JohnOmernik]

So I've been playing with Calico running on Mesos, and with Marathon, the IP per container stuff is well integrated and "easy" (once you get things setup, the getting it setup needs a bit work, but I digress).

That said, It would be outstanding if like Marathon, we could specify to Mesos to use a calico/overlay network for the brokers (and the scheduler, but that would be applied int he Marathon definition). I wish I knew how "hard" this was under the hood, but with how Marathon did it's "end result" just is awesome.

John