search

projectcalico / calico

Cloud native networking and network security
https://docs.tigera.io/calico/latest/about/
Apache License 2.0
5.72k stars 1.27k forks source link

Support IP fragmentation in eBPF #8821

Open nick-oconnor opened 1 month ago

nick-oconnor commented 1 month ago

Expected Behavior

UDP packet fragments destined for a pod's IP which are not denied by policy arrive on the pod's interface.

Current Behavior

The eBPF data plane appears to be dropping UDP packet fragments by policy. The initial fragment is correctly forwarded from the node interface to the pod interface, but subsequent fragments do not appear on the pod's interface. When a UDP packet fragment is dropped, calico's dropped by policy counter for the interface is incremented. The pod interface eventually responds with "fragment reassembly time exceeded".

The only policies I have defined are k8s network policies. This problem does not occur when using the IPTables data plane.

Possible Solution

No idea. There may be a bug in calico's eBPF policy code.

Steps to Reproduce (for bugs)

  1. Enable the eBPF data plane (kube-proxy not running, with or without DSR)
    • BGP w/ no encapsulation + dual stack (I'm unsure if this is relevant, packet captures were all IPv4)
  2. Deploy a pod
  3. Start a packet capture on the node running the pod
  4. Send a fragmented UDP packet to the pod IP (I'm unsure how to replicate this outside of SNMP)

Context

I experienced this behavior after migrating from the IPTables data plane to the eBPF data plane. All SNMP responses exceeding the network's MTU caused my SNMP collector to timeout. I used captures from various points to determine where the packets were being dropped.

Your Environment

tomastigera commented 1 month ago

That is correct observation. Unfortunately, ebpf dataplane does not support ip fragmentation as only the first fragment contains udp ports. The subsequent fragments cannot be matched reliably with the ongoing flow. We cannot reassemble the fragments in eBPF easily (that is a limitation of the technology). This said, we might consider some improvements/workarounds in a future release.

nick-oconnor commented 1 month ago

@tomastigera Wow thanks for the quick reply! Very interesting. Looks like I have some homework regarding eBPF APIs. It'll probably save folks some time by adding this to the eBPF docs for Calico.

nick-oconnor commented 1 month ago

Related: https://github.com/cilium/cilium/issues/25709#issuecomment-2105977944

tomastigera commented 1 month ago

Thanks for the pointer. Problem with kfunc is that they are in "newer" kernels only and are not necessarily a stable API. But we could perhaps add it for kernels that have that feature! :+1:

tomastigera commented 1 month ago

Seems like the patch :arrow_up: is not present in any released kernel :(