Currently, coil-egress uses the fixed encap-sport 5555, and it causes some issues.
Underlying network components, e.g., Cilium(eBPF), don't have a chance to pick a new backend during the graceful termination process. They always select the same backend until the backend finally gets removed.
Cilium sometimes wrongly tracks connections for the packets sent from client pods that rely on the egress pod in the opposite direction, breakings communications between client pods and egress pods. Reply packets from the egress pod rely on the rev SNAT and the connection needs to be tracked correctly.
How
Investigate the tunnel collect metadata mode and use it to lift the limitation described above.
What
Currently, coil-egress uses the fixed encap-sport 5555, and it causes some issues.
How
Investigate the tunnel collect metadata mode and use it to lift the limitation described above.
Checklist