search

opnsense / core

OPNsense GUI, API and systems backend
https://opnsense.org/
BSD 2-Clause "Simplified" License
3.35k stars 752 forks source link

Traffic routed arbitrarily over the Wireguad Interface despite disabled WG gateway #5592

Closed wrobelda closed 2 years ago

wrobelda commented 2 years ago

Important notices

Describe the bug

I am migrating my setup from pfSense to OPNSense. Everything was OK so far, until Wireguard client VPN migration. I copied my config 1:1 from pfSense, which was a basic "client" connection to a remote VPN provider, accompanied by a selective traffic redirection for one of the LAN hosts. Used this guide and it worked from scratch: https://docs.netgate.com/pfsense/en/latest/recipes/wireguard-client.html

Now, with OPNSense, here's what's happening:

  1. With my LAN host redirecting rule enabled, I am not getting anything on the host. Checked wg0 interface on the firewall and seeing monitoring ICMP packets only.
  2. After a reboot, now somehow all WAN traffic is routed via wg0, although LAN hosts don't get the Internet (probably because of NAT is somehow messed up)
  3. Since all WAN traffic is being routed, I naturally assumed that the WG gw must have taken over the WAN one. However:
    • WG gw has a lower priority (255) vs the WAN gateway (254)
    • WG gw is not marked as upstream, but WAN gw is
  4. Still, I disabled the WG gw altogether, yet the traffic still shows on the wg0 interface (!), and the LAN hosts are still not connecting to the Internet
  5. So I rebooted, and it's still the same (!!!). Gateway is down, all custom firewall rules disabled, yet the WAN traffic still shows on wg0 :o
  6. Only after disabling wg0 interface things actually got back to normal. Rebooting and re-enabling the interface doesn't bork it up, which is a clear indicator that enabling/disabling things (gateways?) is currently not deterministic.

FYI, I can reproduce this each time.

Expected behavior

– Should be able to selectively route the traffic over the Wireguard gateway. – Gateway should respect the priority and upstream denotation. – Disabling a gateway should revert back to the next one in priority – Disabling a gateway should clean up (revert) all changes done to networking configuration

Environment

Software version used and hardware type if relevant, e.g.:

OPNsense 22.1.1_3-amd64

wrobelda commented 2 years ago

Additionally, I have "Allow default gateway switching" switched off.

mimugmail commented 2 years ago

I'd suggest to open a thread in the forums. Most of the devs only test against site2site and not for VPN providers. I know many guys in the forums using such a setup in production and might better help than us.

wrobelda commented 2 years ago

I've noticed that this was marked with "support" label, but wouldn't the behavior after disabling the gateway be of concern and an indication of a bug? There's clearly something odd happening here, regardless of the VPN setup.

P.S. I added a post on the forums, https://forum.opnsense.org/index.php?topic=27158.0

wrobelda commented 2 years ago

@mimugmail can this please be escalated to a bug from support issue and prioritized?

I just upgraded to 2.1.2 and my LAN hosts lost Internet connectivity. I rebooted once again and noticed it was there for a while before going off within seconds, so I suspected this was somehow related to Wireguard again, and, bingo: despite the WG interface being explicitly disabled in the UI, I can see the wg0 interface is up in the ifconfig and all the WAN traffic is routed via it. Enabling it in the UI and disabling again restored the Internet to LAN hosts.

I have literally just moved from pfSense and in my past 3 years experience have not experienced anything like this... But, in particular, credit where it is due, they treat their issues seriously. Feel like I am being ignored here, on the other hand.

mimugmail commented 2 years ago

I'll answer in the Forums.

OPNsense-bot commented 2 years ago

This issue has been automatically timed-out (after 180 days of inactivity).

For more information about the policies for this repository, please read https://github.com/opnsense/core/blob/master/CONTRIBUTING.md for further details.

If someone wants to step up and work on this issue, just let us know, so we can reopen the issue and assign an owner to it.