MTU issues with distributed emulation

Here is a note if you are distributing your emulation across several CORE 
servers.

We've noticed MTU issues, caused by the GRE devices used by CORE to build 
tunnels between servers. The gretap device has an interface MTU of 1458 bytes. 
When you join a gretap to a Linux bridge, the MTU for the bridge becomes 1458 
bytes. Other bridge ports may have veths with MTU 1500. The bridge will not 
perform fragmentation for large packets arriving on a port with a higher MTU 
than the destination port. Packets are silently dropped.

Below is an example of a virtual node pinging across a GRE tunnel to a virtual 
node on another emulation server:

root@n2:/tmp/pycore.56151/n2.conf# ping -s 1430 10.0.1.2
PING 10.0.1.2 (10.0.1.2) 1430(1458) bytes of data.
1438 bytes from 10.0.1.2: icmp_req=1 ttl=64 time=0.667 ms
1438 bytes from 10.0.1.2: icmp_req=2 ttl=64 time=0.662 ms
^C
--- 10.0.1.2 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.662/0.664/0.667/0.025 ms
root@n2:/tmp/pycore.56151/n2.conf# ping -s 1431 10.0.1.2
PING 10.0.1.2 (10.0.1.2) 1431(1459) bytes of data.
^C
--- 10.0.1.2 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2016ms

root@n2:/tmp/pycore.56151/n2.conf#

The larger ping packet that exceeded 1458 bytes was dropped. The workaround for 
this to set all interfaces connected to a bridge having a gretap to the smaller 
MTU size. This will cause the large packets to properly fragment and be 
delivered:

ip link set mtu 1458 dev eth0
ip link set mtu 1458 dev eth1

(Don't forget to set MTUs for interfaces on both sides of the tunnel.)

Original issue reported on code.google.com by ahrenh...@gmail.com on 16 Dec 2011 at 4:43

gregtampa / coreemu

MTU issues with distributed emulation #93