Server Configuration: Routing Traffic to WireGuard Interface

davidstrt commented 4 weeks ago

Operating system

Linux

System version

Ubuntu 20.04, Kernel: 5.15.0-107-generic

Installation type

Original sing-box Command Line

If you are using a graphical client, please provide the version of the client.

No response

Version

sing-box version 1.9.0

Environment: go1.22.3 linux/amd64
Tags: with_gvisor,with_quic,with_dhcp,with_wireguard,with_ech,with_utls,with_reality_server,with_acme,with_clash_api
Revision: 5ff7006326e8a876d33d92b26ebd2671cdd48b9f
CGO: disabled

Description

I'm currently trying to route traffic from Sing-Box server to a WireGuard interface ( wg0 ), on the same machine ( VPS 1 ). I've noticed that when the wg0 interface is down, I can use outbounds with "type": "wireguard" without any issues ( using sing_out_wg.json config ). However, as soon as I enable the wg0 interface using wg-quick up wg0 and using sing_out_direct.json config, my Sing-Box clients stop working.

   PC / Mobile          VPS 1 ( sing-box server / wg client )                VPS 2                        
   ┌──────────┐       ┌───────────────────────────────────────┐       ┌──────────────────┐                
   │  NOT     │       │ > wg-quick up wg0                     │       │                  │                
 ┌─┴┐ Working │ ────► │                                       │ ────► │ wireguard-server │ ─────► Internet
 │  ├─────────┘       │ > sing-box run -c sing_out_direct.json│       │                  │                
 └──┘                 └───────────────────────────────────────┘       └──────────────────┘                

   ┌──────────┐       ┌───────────────────────────────────────┐       ┌──────────────────┐                
   │          │       │ > wg-quick down wg0                   │       │                  │                
 ┌─┴┐ Working │ ────► │                                       │ ────► │ wireguard-server │ ─────► Internet
 │  ├─────────┘       │ > sing-box run -c sing_out_wg.json    │       │                  │                
 └──┘                 └───────────────────────────────────────┘       └──────────────────┘

I've tried setting "bind_interface": "wg0" on the direct outbound, but this doesn't seem to establish a connection either.

I'm wondering if there's any additional routing configuration needed in addition to the "bind_interface" config? Or is this configuration even relevant in this context?

Any guidance or suggestions would be greatly appreciated.

P.S: If you're curious why I need the reroute config, it's because the WireGuard server alters its credentials as soon as the interface goes offline, which necessitates directing traffic to the WireGuard interface.

Reproduction

VPS 1 ( Sing-box Server / Wireguard Client ):

wg showconf wg0

```conf [Interface] ListenPort = 33986 FwMark = 0xca6c PrivateKey = eIRLTVt73RComsJEL9jwhMWrz7zjnIi97XS7gRGCuG8= [Peer] PublicKey = YUTsPMYJfrqXjIFmYXOeKWV/WRajm/uDC5e+1XM2c2M= AllowedIPs = 0.0.0.0/0 Endpoint = :51820 PersistentKeepalive = 25 ```

wg-quick up wg0

``` [#] ip link add wg0 type wireguard [#] wg setconf wg0 /dev/fd/63 [#] ip -4 address add 10.10.10.2/24 dev wg0 [#] ip link set mtu 1420 up dev wg0 [#] resolvconf -a tun.wg0 -m 0 -x [#] wg set wg0 fwmark 51820 [#] ip -4 route add 0.0.0.0/0 dev wg0 table 51820 [#] ip -4 rule add not fwmark 51820 table 51820 [#] ip -4 rule add table main suppress_prefixlength 0 [#] sysctl -q net.ipv4.conf.all.src_valid_mark=1 [#] iptables-restore -n ```

cat sing_out_direct.json

```json { "inbounds": [ { "type": "vmess", "tag": "vmess-in", "listen": "::", "listen_port": 11144, "tcp_fast_open": true, "users": [ { "name": "vmessuser", "uuid": "d753ade4-0efe-4d86-bb5b-41f86aeb1e48", "alterId": 0 } ], "tls": { "enabled": true, "server_name": "www.python.org", "reality": { "enabled": true, "handshake": { "server": "www.python.org", "server_port": 443 }, "private_key": "cAl8_5zyUM4Ia4FcgV7WnqLGvhUgKEk_nQKTLnpevWY", "short_id": [ "494b44192830c2c6" ] } }, "multiplex": { "enabled": true, "padding": true, "brutal": { "enabled": true, "up_mbps": 3072, "down_mbps": 3072 } } } ], "outbounds": [ { "type": "direct", "tag": "direct", "bind_interface": "wg0" } ] } ```

sing-box run -c sing_out_direct.json

```bash INFO[0000] router: updated default interface eth0, index 2 INFO[0000] inbound/vmess[vmess-in]: tcp server started at [::]:11144 INFO[0000] sing-box started (0.00s) ```

Client: 🚫🚫🚫 The clients are unable to establish a connection.

wg-quick down wg0

``` [#] ip -4 rule delete table 51820 [#] ip -4 rule delete table main suppress_prefixlength 0 [#] ip link delete dev wg0 [#] resolvconf -d tun.wg0 -f [#] iptables-restore -n ```

cat sing_out_wg.json

```json { "inbounds": [ { "type": "vmess", "tag": "vmess-in", "listen": "::", "listen_port": 11144, "tcp_fast_open": true, "users": [ { "name": "vmessuser", "uuid": "d753ade4-0efe-4d86-bb5b-41f86aeb1e48", "alterId": 0 } ], "tls": { "enabled": true, "server_name": "www.python.org", "reality": { "enabled": true, "handshake": { "server": "www.python.org", "server_port": 443 }, "private_key": "cAl8_5zyUM4Ia4FcgV7WnqLGvhUgKEk_nQKTLnpevWY", "short_id": [ "494b44192830c2c6" ] } }, "multiplex": { "enabled": true, "padding": true, "brutal": { "enabled": true, "up_mbps": 3072, "down_mbps": 3072 } } } ], "outbounds": [ { "type": "wireguard", "tag": "wg-out", "server": "", "server_port": 51820, "system_interface": false, "gso": false, "interface_name": "wg1", "local_address": [ "10.10.10.2/24" ], "private_key": "eIRLTVt73RComsJEL5jwhMWrz7zjnIi97XS7gRGCuG8=", "peer_public_key": "YUTsPMYJfrqXjIFmYXOeKWV/WRajm/uDC1e+1XM5c2M=", "mtu": 1420 } ] } ```

sing-box run -c sing_out_wg.json

```bash INFO[0000] router: updated default interface eth0, index 2 DEBUG[0000] outbound/wireguard[wg-out]: uapi: updating private key DEBUG[0000] outbound/wireguard[wg-out]: peer(YUTs…2c2M) - uapi: created DEBUG[0000] outbound/wireguard[wg-out]: peer(YUTs…2c2M) - uapi: updating endpoint DEBUG[0000] outbound/wireguard[wg-out]: peer(YUTs…2c2M) - uapi: adding allowedip INFO[0000] inbound/vmess[vmess-in]: tcp server started at [::]:11144 INFO[0000] sing-box started (0.22s) DEBUG[0000] outbound/wireguard[wg-out]: routine: encryption worker 1 - started DEBUG[0000] outbound/wireguard[wg-out]: routine: decryption worker 1 - started DEBUG[0000] outbound/wireguard[wg-out]: routine: handshake worker 1 - started DEBUG[0000] outbound/wireguard[wg-out]: routine: tun reader - started DEBUG[0000] outbound/wireguard[wg-out]: routine: event worker - started DEBUG[0000] outbound/wireguard[wg-out]: interface up requested DEBUG[0000] outbound/wireguard[wg-out]: udp bind has been updated DEBUG[0000] outbound/wireguard[wg-out]: peer(YUTs…2c2M) - starting DEBUG[0000] outbound/wireguard[wg-out]: interface state was Down, requested Up, now Up DEBUG[0000] outbound/wireguard[wg-out]: peer(YUTs…2c2M) - routine: sequential receiver - started DEBUG[0000] outbound/wireguard[wg-out]: routine: receive incoming receive - started DEBUG[0000] outbound/wireguard[wg-out]: peer(YUTs…2c2M) - routine: sequential sender - started ```

Client: ✅✅✅ Connection can be established by the clients.

Logs

No response

Supporter

[ ] I am a sponsor

Integrity requirements

[X] I confirm that I have read the documentation, understand the meaning of all the configuration items I wrote, and did not pile up seemingly useful options or default values.
[X] I confirm that I have provided the server and client configuration files and process that can be reproduced locally, instead of a complicated client configuration file that has been stripped of sensitive data.
[X] I confirm that I have provided the simplest configuration that can be used to reproduce the error I reported, instead of depending on remote servers, TUN, graphical interface clients, or other closed-source software.
[X] I confirm that I have provided the complete configuration files and logs, rather than just providing parts I think are useful out of confidence in my own intelligence.

nekohasekai commented 3 weeks ago

ip -4 rule add not fwmark 51820 table 51820 hijacks all connections, change your config to remove that rule.

davidstrt commented 3 weeks ago

Great, Works like a charm :)

SagerNet / sing-box