tailscale / tailscale

The easiest, most secure way to use WireGuard and 2FA.
https://tailscale.com
BSD 3-Clause "New" or "Revised" License
16.84k stars 1.27k forks source link

cmd/containerboot: reaper races with iptables commands to reap child processes #11893

Closed irbekrm closed 2 weeks ago

irbekrm commented 2 weeks ago

What is the issue?

After containerboot has started up, it runs a loop waiting on any child process state changes.

Containerboot can configure firewall/proxy rules, using iptables via coreos/go-iptables library and in some cases, when some state has changed, the iptables firewall gets reconfigured at some point post-startup. coreos/go-iptables just shells out to iptables CLI via Go's cmd.Run that starts a child process and waits for it to change state.

In practice this means that containerboot's reaper races with cmd.Run started by coreos/go-iptables to clean up the child process once the iptables command has finished and when the reaper's wait syscall runs first and cleans up the child process (which seems to be every time in my tests), the wait syscall started by cmd.Run fails with ECHILD (no child process found, see docs) and the whole command fails causing the container to crash:

..
boot: 2024/04/27 12:48:40 installing egress proxy rules for destination 100.73.34.102: installing egress proxy rules: wait: no child processes
...

We added the reaper in https://github.com/tailscale/tailscale/pull/6658, discussed whether it's needed in https://github.com/tailscale/tailscale/pull/10035#discussion_r1381543051

Steps to reproduce

To reproduce with a kube operator egress proxy:

irbekrm commented 2 weeks ago

Fixed in #11897