search

weaveworks / ignite

Ignite a Firecracker microVM
https://ignite.readthedocs.org
Apache License 2.0
3.49k stars 228 forks source link

Support for ipvlan or macvlan? #810

Open gaby opened 3 years ago

gaby commented 3 years ago

I was wondering if there's way to run ignite microVM's using Docker ipvlan/macvlan?

Usually the way I go about this is:

docker network create -d ipvlan --subnet=10.0.2.0/24 -o parent=enp0s3.2 vlan_2
docker network create -d ipvlan --subnet=10.0.4.0/24 -o parent=enp0s3.4 vlan_4

docker run -it --rm --net=vlan_2 --ip=10.0.2.15 alpine ipconfig
docker run -it --rm --net=vlan_4 --ip=10.0.4.15 alpine ipconfig

No only this setup gives you static-ips on each container, you also get L2 isolation via vlan trunking. Looking at the help menu from ignite the only option is "network-plugin" which only supports "docker-bridge".

stealthybox commented 3 years ago

@luxas and @bboreham are currently pairing on a macvlan implementation. Some of the bits and pieces are working.

The tricky bits here are how the network namespace and device nodes are setup in the ignite-spawn sandbox container because a character device needs to be passed through the firecracker config if we are not using TAP's with /dev/net/tun.

Keeping in mind the Docker network-plugins, we would have to look into how the sandbox can look and what additional work needs to occur to expose those interfaces to the firecracker guest -- this problem has additional requirements beyond normal container networking.

gaby commented 3 years ago

I'm not sure how doable it would be but, attaching an ignite vm's to a docker networks? Ignite currently supports using the docker-bridge, would it be much different to use an user-created network?

bboreham commented 3 years ago

Small clarification: we were working on macvtap.

Ipvlan will not work with the current code; it expects that you give it a device which can has its own L2 identity. I made #761 to fix the obvious crash but I suspect it won't work even after that.

As for using Docker network, I don't see any fundamental obstacle - it's doing roughly what the current CNI-based code does - but for these things the devil is very much in the detail. Docker does many things in pursuit of its "virtual service" concept that are unlikely to play well with what a VM needs.