jepsen-io / jepsen

A framework for distributed systems verification, with fault injection
6.82k stars 718 forks source link

Why delete all iptables chains? #600

Closed wb14123 closed 2 weeks ago

wb14123 commented 2 weeks ago

At here: https://github.com/jepsen-io/jepsen/blob/9e4987a95040ce82a2bf3a7e3668a1164bccf64b/jepsen/src/jepsen/net.clj#L188, Jepsen deletes all the iptables chains during net heal. Is it necessary? I only find Jepsen modifying INPUT chain so seems just flush the chains is enough?

Asking this because Kubernetes created a bunch of iptables chains and are failed to delete because of error:

iptables v1.8.9 (nf_tables):  CHAIN_DEL failed (Device or resource busy): chain KUBE-ROUTER-INPUT 
aphyr commented 2 weeks ago

The short answer is: it's easy. Jepsen assumes it's the only thing that touches these nodes. If you'd like to modify the iptables commands so they do something more clever--say, deleting specific rules that previous Jepsen runs might have created, go for it! The major factors are: 1.) avoid round trips--we want to do this with as little latency as possible, and 2.) clean recovery across runs and versions. We want to be able to clean up whatever partitions a previous test run, possibly running a different version of Jepsen, left behind, even if it crashed halfway through.On Oct 21, 2024 20:47, Bin Wang @.***> wrote: At here: https://github.com/jepsen-io/jepsen/blob/9e4987a95040ce82a2bf3a7e3668a1164bccf64b/jepsen/src/jepsen/net.clj#L188, Jepsen deletes all the iptables chains during net heal. Is it necessary? I only find Jepsen modifying INPUT chain so seems just flush the chains is enough? Asking this because Kubernetes created a bunch of iptables chains and are failed to delete because of error: iptables v1.8.9 (nf_tables): CHAIN_DEL failed (Device or resource busy): chain KUBE-ROUTER-INPUT

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.***>

wb14123 commented 2 weeks ago

Got it. I can only delete iptables rules in my own Net implementation. Just wanted to confirm there is no other reasons I'm not aware of that I cannot do so. Thanks!