weaveworks / ignite

Ignite a Firecracker microVM
https://ignite.readthedocs.org
Apache License 2.0
3.49k stars 224 forks source link

DHCP sending corrupt UDP checksum #870

Open networkop opened 2 years ago

networkop commented 2 years ago

Hi friends šŸ‘‹ , hope y'all are doing well. It's me again with a minor bug we've picked up recently. It looks like the tap interfaces that we create for FC VM have checksum offloading enabled by default, which leads to DHCP reply packets reaching the eth0 inside a VM with incorrect checksum.

For some reason, older kernel versions do not ignore this and drop the DHCP replies instead of accepting them:

Sep 15 10:42:38 5557ffdad223d394 dhclient[3259]: 5 bad udp checksums in 5 packets
Sep 15 10:42:46 5557ffdad223d394 dhclient[3259]: No DHCPOFFERS received.

See this issue for more details: https://github.com/networkop/cx/issues/5

I've confirmed that changing the TX offset to off on vm_eth0 fixes the issue and eth0 can successfully obtain the DHCP IP.:

 ethtool --offload vm_eth0 tx off

We can do this for all FC interfaces, since offloading does not help in the case of a virtual NIC. We can either shell out and run this as a command or use this function to change at the cost of an additional dependency.

wdyt?