Alternatives to macvlan for network virtualization?

[ghost]

From my very coarse understanding of macvlan, it doesn't seem possible to use it if the adapter is virtualized or otherwise allow for multiple MAC addresses (e.g. on VMWare or a cloud hosting provider).

Either I misunderstand how to configure macvlan in these scenarios (and I hope I do!), or is there an alternative to macvlan for forwarding network packets to the jail?

[ghost]

Just a small followup: it seems like veth might be a good alternative to macvlan in this scenario.

[robertswiecki]

@q3k was interested in cleaning-up this area. Serge, do you still have any plans to work on that? If not, I might take a look at some point in the future.

@rfw If you feel strongly about that, feel free to propose some ideas or/and code

[ghost]

I've managed to kludge around it by creating a veth interface and adding a macvlan on the veth interface's peer – it works, but it's not pretty.

I took a stab at changing the code to support veth directly but wasn't sure how you'd like the command line interface to look for different network types.

[robertswiecki]

What about starting with the config file (in protobuf). Maybe creating a message (subsection) like Networking would be good?

[SJLC]

Is anybody working on a branch with these proposed changes somewhere? I'm considering adding a --phys_interface option for the (admittedly less common) case where there is a spare physical interface available to dedicate to a server running in a container, and don't want to have a bunch of conflicts with similar changes being worked on elsewhere.

[q3k]

Some time ago I was thinking of ripping out SLIRP code from QEmu to create a userspace NAT for nsjail containers. However, I haven't yet had time to do this yet, especially as that code is quite ugly (and the licensing issue is nontrivial, too). I then yakshaved this into thinking of recreating a SLIRP alternative. And then I left Google :).

If anyone wants to take this on, I wouldn't mind - it seems like a fairly fun and self-contained project. Otherwise I might take a look some day.

[andrewchambers]

For anyone out there, currently worked around this issue by using --disable_clone_newnet . Example:

      # This whole netns and veth thing is a 
      # workaround because nsjail doesn't support
      # veth yet, see https://github.com/google/nsjail/issues/20
      ip link set dev boxveth1 down || true
      ip link delete boxveth1 || true
      ip netns delete boxns || true

      ip link add boxveth1 type veth peer name boxveth2
      ip netns add boxns
      ip link set boxveth2 netns boxns

      ip link set dev boxveth1 up
      ifconfig boxveth1 10.1.1.1/24 up

      ip netns exec boxns ip link set dev boxveth2 up
      ip netns exec boxns ifconfig boxveth2 10.1.1.2/24 up

      exec ip netns exec boxns nsjail --disable_clone_newnet ...

[robertswiecki]

Maybe

--iface_own

will work for you? It doesn't need --disable_clone_newnet, but will require running from root.

The idea is to create a veth pait, and then

nsjail --iface_own [one_of_veths]

[andrewchambers]

@robertswiecki Ah, thank you! my package manager was using an old version, updating gave me this option.