Multus MacVlan on Windows

[maximehyh]

Hi,

I am trying to setup a K8s cluster on a Windows host using multipass and microk8s (https://multipass.run/docs/installing-on-windows). The goal is for my pod to connect to a camera that uses the GigE protocol and (to summarize) requires that my pod has "full access" to the camera network.

I used to do this on Docker Swarm on Linux using MacVLANs on a physical network interface but I would like to do the same thing on a windows machine. The tricky thing is that my Kubernetes now runs in a VM as opposed to my Swarm cluster that was running (directly) on a Linux host.

Multipass (through Hyper-V) allows to make a virtual interface in order to bridge the VM to the network (See network=bridge https://multipass.run/docs/launch-command). Which gives me an interface with the same DHCP IP as the (windows) host, and that I can ping/reach from outside.

I figured that I could use this same interface to use as a master in my NetworkAttachmentDefinition in order to make a MacVlan in Kubernetes. I successfully built the network, pods connected to the newly created network can ping each other using the provided IP, but I cannot succeed to ping those pods either from the multipass VM, from the windows host, and of course from another machine on the network.

Is there a way to build a MacVLAN using a virtual interface? And have you seen any cases of people trying to build a MacVLAN in a VM on Windows?

Below my NetworkAttachmentDefinition.

Thanks!

apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: my-network
spec:
  config: '{
    "cniVersion": "0.3.0",
    "name": "my-network",
    "type": "macvlan",
    "master": "eth1",
    "mode": "bridge",
     "ipam": {
      "type": "host-local",
      "ranges": [
         [ {
           "subnet": "192.168.0.0/24",
           "rangeStart": "192.168.0.40",
           "rangeEnd": "192.168.0.45",
           "routes": [ { "dst": "0.0.0.0/0" } ],
           "gateway": "192.168.0.1"
         } ]
      ]
    }
  }'

VM Network:

PS C:\Users\User>  multipass exec VM_NAME -- ip a
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether XX:XX:XX:XX:XX brd ff:ff:ff:ff:ff:ff
    inet 172.27.194.25/20 brd 172.27.207.255 scope global dynamic eth0
       valid_lft 81323sec preferred_lft 81323sec
    inet6 XXXX::XXXX:ff:XXXX:XXXX/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether XX:XX:XX:XX:XX:XX brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.30/24 brd 192.168.0.255 scope global dynamic eth1
       valid_lft 599723sec preferred_lft 599723sec
    inet6 XXXX::XXXX:ff:XXXX:XXXX/64 scope link
       valid_lft forever preferred_lft forever

Windows interfaces

PS C:\Users\mUser> Get-NetAdapter

Name                      InterfaceDescription                    ifIndex Status       MacAddress             LinkSpeed
----                      --------------------                    ------- ------       ----------             ---------
vEthernet (WSL)           Hyper-V Virtual Ethernet Adapter #2          40 Up           XX-XX-XX-XX-XX-XX        10 Gbps
vEthernet (ExtSwitch (Wi… Hyper-V Virtual Ethernet Adapter #3          28 Up           XX-XX-XX-XX-XX-XX        780 Mbps
Network Bridge            Microsoft Network Adapter Multiplexor …      16 Up           XX-XX-XX-XX-XX-XX        780 Mbps
Wi-Fi                     Intel(R) Wi-Fi 6E AX210 160MHz               14 Up           XX-XX-XX-XX-XX-XX        780 Mbps
Bluetooth Network Connec… Bluetooth Device (Personal Area Networ…      13 Disconnected XX-XX-XX-XX-XX-XX         3 Mbps
vEthernet (Default Switc… Hyper-V Virtual Ethernet Adapter             27 Up           XX-XX-XX-XX-XX-XX        10 Gbps

PS C:\Users\User> multipass list
Name                    State             IPv4             Image
VM_NAME      Running           172.27.194.25    Ubuntu 20.04 LTS
                                          192.168.0.30
                                          10.1.42.128

[dougbtv]

Hi @maximehyh ! We think your problem would probably be better served by the reference CNI plugin maintainers who may have more in depth knowledge for macvlan in windows: https://github.com/containernetworking/plugins/issues

[maximehyh]

Ok Thanks