`set-default-gateway` blocking routing issue

[lrodorigo]

I noticed that setting the set-default-gateway: true option creates a sort of "routing hole" since the vpn packets are routed via the "default gateway" (which is the VPN itself), the first "Ping" is lost and the VPN disconnects.

There are multiple ways to solve the issue, such as:

1) Add a specific route (after the DNS resolution of the server): ip route add <SERVER IP>/32 via <PREVIOUS_DEFAULT_GW>

Basic idea:

#!/bin/bash
TARGET_HOST="wsvpn.myhost.com"
VPN_IP="192.168.9.1"
while true; do
    TARGET_HOST_IP=$(nslookup "$TARGET_HOST" | grep -A1 "Name:" | tail -n 1 | awk '{print $2}')
    OLD_GW=$(route -n | grep -v $VPN_IP | grep "0.0.0.0" | head -n 1 | awk '{print $2}')

    ip route add $TARGET_HOST_IP via $OLD_GW

    ./wsvpn-linux-amd64 --mode client --config client.yaml

    echo "VPN Disconnected - Deleting route"

    ip route del $TARGET_HOST_IP || /bin/true

    sleep 5

done

2) Fw Mark + IP tables (look at Wireguard docs https://www.wireguard.com/netns/, section Routing All Your Traffic)

NOTE: An additional issue when using the automatic reconnection is that, after the first disconnection, the DNS resolution is also routed on the (disconnected) VPN gateway, so after the first disconnection, the connections drops forever and it is not able to connect again.

[Doridian]

For 2: Wireguard's connection marking code seems to be implemented in Go here: https://git.zx2c4.com/wireguard-go/tree/conn/mark_unix.go I should be able to make a similar implementation in WSVPN.

There is also 3: Run the VPN as a specific dedicated UID and use iptables UID filtering (-m uid --uid-owner X matcher)

[lrodorigo]

Good catch for IP Tables.

Packet marking it is a really neat solution... I would like to contribute on this fantastic project, but my Go knowledge is quite limited, sorry.

Just an additional note: consider that when the automatic reconnection is enabled, the DNS resolution of the server address must be performed using the "previous" default-gateway...

[Doridian]

Firewall marking has now been released in v5.35.0 :) There isn't much more that can be done from WSVPN's end so I will be closing this issue.

[lrodorigo]

I think that when using the default GW option, wsvpn itself should setup the right ip rules, in order to route all the non-marked traffic on the vpn gateway and all the marked traffic on the old default GW... or an handler.sh script could be provided.

Moreover: is also the DNS resolution traffic marked? I think that is better to manually resolve the server hostname before changing the default GW.

[Doridian]

@lrodorigo For WSVPN to actually set rules/routing itself, it would require a lot more logic, such as determining what the default gateway interface is. I do not think I have the time available to spearhead such an effort.

As for the DNS traffic issue: WSVPN only reroutes the gateway once it has successfully connected, therefor DNS traffic should not need to be marked. (Furthermore, in most cases your DNS server should be on your LAN, which has a more specific route and therefor does not get rerouted)

[lrodorigo]

Okay I will try to contribute by providing an example handler. The previous gateway it is not strictly needed since all the non marked traffic must be forwarded to the wsvpn gateway (which is known).

I think that the DNS traffic issue arise only when the automatic reconnection mode is enabled.

Il dom 7 gen 2024, 23:52 Mark Dietzer @.***> ha scritto:

@lrodorigo https://github.com/lrodorigo For WSVPN to actually set rules/routing itself, it would a) Require to be run as root, which I do not want to require b) Require a lot more logic, such as determining what the default gateway is. I do not think I have the time available to spearhead such an effort.

As for the DNS traffic issue: WSVPN only reroutes the gateway once it has successfully connected, therefor DNS traffic should not need to be marked.

— Reply to this email directly, view it on GitHub https://github.com/Doridian/wsvpn/issues/435#issuecomment-1880204581, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACBHPUHO5KM7BMNU4LABZ73YNMRKJAVCNFSM6AAAAABBQF5Q7KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBQGIYDINJYGE . You are receiving this because you were mentioned.Message ID: @.***>

[Doridian]

@lrodorigo Yeah, fair point. Maybe we could modify the ip route commands to exclude marked traffic, or only add rotues for non-marked traffic. I didn't work with those rules in a while.

As for the DNS issue, yeah that makes sense with reconnection. I could try marking DNS traffic as well, but WSVPN currently does not handle DNS resolutions on its own, so that would have to be a change with some involvement as well.

[lrodorigo]

I know that handling manual DNS resolution is cumbersome, at this point an external bash script could do the job, and you can make mutually exclusive the automatic reconnection and the default gateway options. It is important that the routes are properly reverted on wsvpn termination.

https://ro-che.info/articles/2021-02-27-linux-routing Well written doc about the wireguard routing and the handling of the ip rule table and prefix length suppressor.

[Doridian]

@lrodorigo I found a way to do this in Golang for DNS as well, see the newest release https://github.com/Doridian/wsvpn/releases/tag/v5.36.0

WSVPN will (or, should) properly revert the routing table once it terminates and deletes its interface.

[lrodorigo]

Great work, I am going to test it as soon as possible.

I am using wsvpn as startup systemd service, I am routing all my network traffic and it is great... Really stable and fast. Often I forgot that I am behind an 'home built' vpn.

I will try to contribute with an example handler script that routes all non marked traffic on the wsvpn gateway.

Many thanks

[Ty9000]

Hello! I wanted to add to this issue, since I'm having the same.

Both server and client: WSVPN version v5.38.2 Server is Fedora 40; client is Ubuntu 24.04 LTS

When setting the client 'set-default-gateway: true', the VPN disconnects after the first hello ping.

And since there is no route to the other internal devices, I must manually add a route to that subnet. (If that's normal behavior, I can just add a script under the scripts section, no biggie!)

Here is the server's yaml:

tunnel:
  mtu: 1420
  subnet: 192.168.3.0/24 # Server will pick the first host from this, and assign others to clients in order
  mode: TAP # TUN or TAP

  # Below settings are only effective when one-interface-per-connection is false/off
  # If you use one-interface-per-connection, use your OS firewall to regulate packet flow
  allow-client-to-client: true # Allow clients to talk to other clients on the same server
  allow-ip-spoofing: true # TAP only: Allow clients to use any IP address and not just the assigned one (on TUN IPs are always enforced)
  allow-mac-changing: true # TAP only: Allow clients to change the MAC address(es) they use on the interface
  allowed-macs-per-connection: 1 # TAP only: Allow multiple MAC addresses per connection
  allow-unknown-ether-types: false # TAP only: Allow unknown ether types (Anything other than ARP, IPv4)
  # Above settings are only effective when one-interface-per-connection is false/off

  features:
    fragmentation: true # Enable packet fragmentation (default enabled), required for MTU > 1216 in WebTransport
  ip-config:
    local: true # Configure local interface automatically
    remote: true # Send configuration data to clients for their interfaces
  ping:
    interval: 25s
    timeout: 5s

interface:
  name: "" # Name of the interface to use, will be used as a prefix is one-interface-per-connection is chosen
  persist: false
  component-id: root\tap0901 # Windows only. Defaults: root\tap0901 or tap0901

  # Warning: This below option will prevent all the tunnel->allow from taking effect. Use iptables as needed!
  one-interface-per-connection: false # Set to true to use separate interface per connection

scripts:
  # These scripts get run as "args... operation subnet interface user"
  # Pass in an array, first argument is the executable, further arguments
  # are used before WSVPN provided arguments
  # User will not be present if no authentication is enabled
  # Example: "./handler.sh" might be called like "./handler.sh up 192.168.3.2/24 tun0 user"
  up: []
  down: []
  # Interface will only be set if the server has "one-interface-per-connection" set to false
  # User will never be set
  startup: []

server:
  listen: ":9000"
  enable-http3: true
  website-directory: "" # Serve normal HTTP(S) requests from this folder, disabled if blank

  headers: # Map of headers (string key to *list* of string values)
  # X-Some-Host:
  #   - example.com
  # X-Other-Header:
  #   - value1
  #   - value2

  tls:
    client-ca: "/opt/private/pki/ca.crt" # Filename of CA for mTLS
    certificate: "/opt/private/pki/issued/vpn.crt" # Filename of certificate for TLS
    key: "/opt/private/pkiKeys/vpn.key" # Filename of private key for TLS
    config:
      min-version: 1.2
      max-version: 1.3
      cipher-preference: "" # blank, AES or CHACHA
      key-log-file: "" # This will log TLS secret keys to a file. DO NOT USE IN PRODUCTION!
#  authenticator:
#    type: allow-all # radius, allow-all or htpasswd
#    # allow-all: Just allow all clients regardless of authentication
#    # htpasswd: Set config key to filename of a htpasswd-formatted file; Authenticates clients using HTTP Basic authentication
#    # radius: Set to the path of a YAML file containing the keys "server: HOST:PORT" and "secret: SHARED_SECRET" 
#    config: ""
  max-connections-per-user: 0 # Only works with a form of authentication enabled, 0 to disable
  max-connections-per-user-mode: kill-oldest # kill-oldest or prevent-new
  api:
    enabled: false # Whether to enable the API
    users: [] # Which users are allowed to use the API. Leaving this empty allows any authenticated user!
  preauthorize-secret: "" # Will enable preauthorization if set

Here is the client's yaml:

tunnel:
  set-default-gateway: false
  ping:
    interval: 25s
    timeout: 5s
  features:
    fragmentation: true # Enable packet fragmentation (default enabled), required for MTU > 1216 in WebTransport

interface:
  name: ""
  persist: false
  component-id: root\tap0901 # Windows only. Defaults: root\tap0901 or tap0901

firewall-mark: 0 # Linux only. Set to positive integer to mark packets with this value in the firewall

scripts:
  # These scripts get run as "args... operation subnet interface"
  # Pass in an array, first argument is the executable, further arguments
  # are used before WSVPN provided arguments
  # Example: "./handler.sh up 192.168.3.2/24 tun0"
  up: []
  down: []

client:
  server: "webtransport://<vpn>:9000" # Examples: ws://example.com:9000 wss://secure.example.com:9000
  proxy: "" # Example: http://user:password@proxy.example.com:8080
  auth-file:  "" # Filename of file containing user:password for HTTP Basic authentication
  auto-reconnect-delay: 0s # Delay after which to retry connecting to the server automatically after an error, set to 0s (default) to disable

  headers: # Map of headers (string key to *list* of string values)
  # Host:
  #   - example.com
  # X-Other-Header:
  #   - value1
  #   - value2

  tls:
    ca: "/root/ca.crt" # Filename of CA bundle for verifying server cert
    certificate: "/root/cert.crt" # Filename of certificate for mTLS
    key: "/root/key.key" # Filename of private key for mTLS
    server-name: "" # If not blank, the hostname to check for in the SSL certificate. If blank, uses hostname from server URL
    config:
      insecure: false
      min-version: 1.2
      max-version: 1.3
      cipher-preference: "" # blank, AES or CHACHA
      key-log-file: "" # This will log TLS secret keys to a file. DO NOT USE IN PRODUCTION!