NICMx / Jool

SIIT and NAT64 for Linux
GNU General Public License v2.0
326 stars 66 forks source link

Low performance on virtual interfaces regardless of GRO #267

Closed ydahhrk closed 5 years ago

ydahhrk commented 6 years ago
  1. Network tool (iperf in this case) writes large packets (> MTU) because it has a large buffer. (Or, alternatively, network tool writes large packets because the kernel is lazy and likes to merge socket data.)
  2. Packets reach virtual interface. Virtual interfaces do not segment the data because they do not care about MTU. (Linux defers data segmentation as much as possible because this actually prevents the segmentation unless it's actually necessary. ("Actually necessary" means that it reaches an actual physical interface.))
  3. Jool sees packet too big, drops. Network tool doesn't know what's going on because it keeps thinking that the packet is going to be segmented in time. Random black hole ensues; TCP retries and only some packets get through. Low performance is observed.

This is similar to the GRO problem, except the overly big data was likely never segmented in the first place.

Thanks to Muhammad Ali for reporting this bug.

ydahhrk commented 6 years ago

I feel like we're closer to fixing the GRO problem, if it isn't fixed already :tada:

ydahhrk commented 6 years ago

Doesn't work yet. It crashes the kernel in some circumstances and still yields low performance in others. Do not use this branch yet.

TheRedTrainer commented 6 years ago

Fixed. Jool doesn't crash the kernel anymore. The performance improved applying the fix, as it can be confirmed on the following results:

Two hosts (3NS):

3NS_joolnscentos.txt

Two hosts (1VM2NS):

1VM2NS_joolcentos.txt

ydahhrk commented 6 years ago

Thanks! Beginning Jool 3.6.0 release.