networkop / meshnet-cni

a (K8s) CNI plugin to create arbitrary virtual network topologies
BSD 3-Clause "New" or "Revised" License
116 stars 27 forks source link

Help - Dropped packets #27

Closed Cerebus closed 3 years ago

Cerebus commented 3 years ago

I don't know if this is a networking issue in my emulation of a problem with meshnet or a problem with kind.

When I ping sweep, some nodes are unreachable. I set rp_filter=0 on the pods, the docker kind container, and the host, and while that cleared up some issues it didn't solve all of them.

If I set up a ping to an unreachable host and start dumping, I can see the ping arrive at the destination. The echo reply gets dropped in transit. The node where it's dropped is where equal-weight paths diverge.

E.g., n0 pinging n5. Requests go n0-n1-n2-n5, replies go n5-n4-n1-n0. If I tcpdump on n1, I can see the reply arrive but it's silently dropped. iface drop stat is 0. log_martians is on but nothing in the logs.

I think this is just me not gr0kking OSPF, but I'm out of ideas.

networkop commented 3 years ago

hey @Cerebus can you add steps to reproduce?

Cerebus commented 3 years ago

I think I figured it out. The Pod comes up with rp_filter enabled. entrypoint was disabling /proc/sys/net/ipv4/conf/all/rp_filter, but the default was enabled and all the ifaces came up that way.

When I turned off rp_filter in the Pod on the iface where the reply was arriving, everything started working.

Is rp_filter something I can control with meshnet? Or do I have to go deep in kind KubeConfig options, or am I stuck setting this on my pods as they boot?

networkop commented 3 years ago

you can either bake it into the image or set it via entrypoint. meshnet has no control over the internal sysctl params.