Open gaopeiliang opened 1 year ago
Setting pmtudisc on tun0 also forcely set DF on tunnel egress packet, this could be a drawback which should be widely tested on different circumstances. In my point of view, applying the fix or adopting tcp_mtu_probe could be better alternatives.
Can you manually set MTU to the correct value? For instance,
apiVersion: projectcalico.org/v3
kind: FelixConfiguration
metadata:
name: default
spec:
ipv6Support: false
ipipMTU: 1400
Can you manually set MTU to the correct value? For instance,
apiVersion: projectcalico.org/v3 kind: FelixConfiguration metadata: name: default spec: ipv6Support: false ipipMTU: 1400
Manually set an safe small mtu value will work well ,that kernel pmtu func will nerver work .....
Now we optimize mul link with SDWAN, some link will send 'Need frag' to change host mtu out of our control ....
We do not want set smallest mtu also .. and it is differect to change host infra global the world ...
@gaopeiliang can test the issue with newer version of Calico, like 3.25? (3.26 will be released soon) I am not saying that newer versions has the fix (Maybe it has, there has been many changes including fixes since 3.13), but using newer version helps to have a more efficient discussion.
@gaopeiliang can test the issue with newer version of Calico, like 3.25? (3.26 will be released soon) I am not saying that newer versions has the fix (Maybe it has, there has been many changes including fixes since 3.13), but using newer version helps to have a more efficient discussion.
It is special kernel version bug about handle PMTU , there is no relation with special calico version , and "init tunl0 device attr" d.dataplane.RunCmd("ip", "tunnel", "add", "tunl0", "mode", "ipip")
never changed !
we lastest tested calico version 3.21 .
Current Behavior
now Felix init tunl0 device when start like this ...
tunl0 device has default nopmtudisc attr;
with nopmtudisc tunl0 can not adaptor mtu change with PMTU discover, and Kernel default set
means IP packet always set DF bit; when host1 -> host2 mtu change small than tunl0 and calixxx because "Need Frargment" ICMP;
tunl0 will drop IP packet because small mtu problem;
the drop IP packet will also produce Need Fragment ICMP self to self, so host1 can not recover mtu forever .......
Possible Solution
it will make tunl0 device update link mtu to correct ...... it will be work OK!
df bool will be set with link attr "pmtudisc" , so tunl0 this attr will affect this case .....
en ,,,, I also find diff kernel version has diff impl...
so before Linux 4.19.161 DF bit only care ip link attr when update mtu, after it will inherit from inner IP packet.... it is an kernel bug to handle mtu ;
fix commit log https://github.com/torvalds/linux/commit/50c661670f6a3908c273503dfa206dfc7aa54c07, the commit msg said case same as this .....
so felix can adaptor this kernel bug when init tunl0 attr pmtudisc when blow kernel version Linux 4.19.161 ......
only some suggests, or have other ideas about this attr ,,,,,
Steps to Reproduce
change host 10.200.xx.196 to host 10.201.xx.21 mtu use ICMP Need Fragment , (image label 3) route cache will be like this
now container 192.168.169.73 hosted on 10.201.xx.21 get data from 192.168.221.15 on hosted 10.200.xx.196 will block because mtu problem ...
Context
tcp connection has mtu problem , make our app random error for a long time ......
randon ICMP change , randon host link mtu , it is too difficulty to debug .....
Environment