Enabling syncookies breaks traffic that both originates and terminates on the firewall

[no-usernames-left]

Important notices

• [X] I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
• [X] I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue

Describe the bug Enabling syncookies breaks traffic both originating and terminating on the firewall (such as when you put Caddy in front of the OPNsense web GUI).

To Reproduce

1. Set up Caddy to proxy the OPNsense web GUI according to the tutorial
2. SSH into the firewall
3. openssl s_client -connect 127.0.0.1:443 (or :8443; both are affected, showing it's neither a Caddy nor a lighttpd issue)
4. See your certificate and be happy
5. Firewall - Settings - Advanced, change Enable syncookies from never (default) to always, click Save
6. openssl s_client -connect 127.0.0.1:443 (or :8443)
7. After connection and a delay of many seconds, see write:errno=54 and be sad

Expected behavior Not this.

Additional context Discovered while troubleshooting this issue.

Environment OPNsense Business 24.4_8 (amd64) os-caddy 1.5.4_1

[AdSchellevis]

I'm not sure this is supposed to work to be honest, local traffic should already have syncookies enabled (net.inet.tcp.syncookies), which might make these two options incompatible. Since pf's syncookies don't have any toggles, it might not be possible to exclude part of the traffic for this specific protection.

[no-usernames-left]

That sounds like a bug to me. It took hours to troubleshoot why Caddy was working fine for all subdomains except OPNsense's web GUI.

[Monviech]

I tried it too and it really happens.

I would have never thought it would be this option. Very obscure. Subscribing to this here.

If its not something that can be fixed I'll add a hint to the docs.

[AdSchellevis]

If it's a bug or a feature really depends on the specifications, which aren't very detailed on this particular subject, I'll move this to src and label it upstream.

At a first glance FreeBSD 14.1 behaves the same in these matters. If pf's syncookies are the right tool to use when the termination point of the traffic is the firewall itself might be a valid question as well (since the tcp stack itself does offer syncookie support anyway). A non stateful inbound rule to prevent state table depletion for a service that terminates on the box might even be more logical choice anyway.

[fichtner]

FWIW, my first thought was upstream bug here as well.

[muratbalaban43]

Let's check with the upstream and see if they can fix it.