hAP2 in NAT mode with WG tunnel, LAN devices can not access anything over WG tunnel, but it can over Wifi/DtD

Marx1 commented 1 week ago

Describe the bug

a hAP2 in NAT mode, any lan devices can not access any devices/nodes across the WG tunnel; however they can access across a RF link.

[ ] Does this issue still occur on the latest version? (Required) Tested on production and nightly.

Expected behavior

WG tunnels act the same from LAN -> WG as LAN -> Mesh/RF

Screenshots

Traceroute from lan device on W6CX-FieldDay-Logging connected to KG6MDW-HAP2 via WG tunnel over t-mobile:

Traceroute from lan device on W6CX-FieldDay-Logging connected to KG6MDW-HAP2 via 5ghz mesh, and t-mobile disconnected (DNS hadn't populated yet):

If applicable, add screenshots to help explain your problem.

Additional context

This sounds more like rule issues that Tim fixed in #1216 I don't have enough brain cycles to dig though the code right now to try and understand why it's happening - I'll see if I can do it later.

Attached are support dumps from both devices.

supportdata-W6CX-FieldDay-Logging-202406132057.tar.gz supportdata-KG6MDW-HAP2-202406131358.tar.gz

aanon4 commented 1 week ago

Could you let me know if this works correctly over a legacy (non-wireguard) tunnel? That'll help me track down the problem.

Marx1 commented 1 week ago

Hi Tim,

I switched over the the legacy tunnel, and at the same time realized KG6MDW-HAP2 was still on production, so I upgraded to nightly as a second check-

Legacy Tunnel does not work:

Neither does Wireguard (verifying with both ends on nightly):

So I'm thinking this is more of a case that there are limited users using NAT mode, and initiating a connection form the LAN targeting a mesh endpoint.

aanon4 commented 1 week ago

NAT mode isn't used much. At least it's consistently broken which makes it easier to work out. Thanks.

Marx1 commented 1 week ago

Np, Thanks for having it. I have a couple of good use cases - for this one the NAT/LAN will be the logging network for Field day, and I want it to be able to access AREDN for resources like NTP/PSK and PBX etc while not exposing all of the devices and log server directly to AREDN.

We use it on Mt Diablo for the same reasons.

aanon4 commented 1 week ago

Another question - does it work across a DTD connection? Looking at the code I think it should only work across wifi right now .. but because that seems slightly unbelievable, can you check? Thanks.

Marx1 commented 1 week ago

it does work across DTD, even before lqm made it active.

aanon4 commented 1 week ago

Any chance you can see the ping packets arrive at the target? I'm wondering if it's the replies which are having trouble.

Marx1 commented 1 week ago

here's via DTD:

here's via WG:

I think you're on to something. It acts like connection tracking in Nat isn't working.

tcpdump on w6cx-fieldday-log WG Interface shows tx and rx:

but br-lan only shows the TX from my bench pc:

aanon4 commented 1 week ago

So the nat firewall configuration has some special rules, and those rules are very old (9-ish years) despite being re-written when the firewall changed. I suspect they were never updated when tunnels were added. Also I think there may be better support in vanilla OpenWRT these days for what these old rules were doing. So I'll see if going that route will fix this, but the changes are a little more complexed.

aanon4 commented 1 week ago

I've sent you a test image for the hAPac2

Marx1 commented 1 week ago

Thanks, I got it installed and tested -

here's over WG:

here's over the legacy tunnel:

doesn't seem to be working

Marx1 commented 1 week ago

Here's it from DTD:

I need to physically move it closer for rf link, but I think this shows it's a all-or-nothing as you suspected.

aanon4 commented 1 week ago

I sent over another test build. Looks like they were messing with the ip rules, so despite the firewall being correct, the reply packets didn't go the right way through the routing tables. I'm really not sure whey they made it this complicated originally.

Marx1 commented 1 week ago

NEW software provided 2:41pm:

Over legacy tunnel: