search

shadowsocks / shadowsocks-rust

A Rust port of shadowsocks
https://shadowsocks.org/
MIT License
8.33k stars 1.14k forks source link

ShadowSocks with Wireguard #785

Closed airled closed 2 years ago

airled commented 2 years ago

Hi. WireGuard is not working via shadowsocks-rust 1.14.1. Server config:

{
  "server": "0.0.0.0",
  "server_port": 14000,
  "password": "pass",
  "timeout": 300,
  "method": "chacha20-ietf-poly1305",
  "mode": "tcp_and_udp"
}

client config:

{
  "server": "server_ip",
  "server_port": 14000,
  "local_address": "0.0.0.0",
  "local_port": 5634,
  "password": "pass",
  "timeout": 300,
  "method": "chacha20-ietf-poly1305",
  "mode": "tcp_and_udp",
  "tunnel_address": "127.0.0.1:51820"
}

Wireguard config:

...
[Peer]
PublicKey = key
Endpoint = 127.0.0.1:5634
AllowedIPs = 0.0.0.0/0

When I up my wireguard interface, shadowsocks logs this error:

2022-03-12T13:47:52.408991642+06:00 INFO  shadowsocks local 1.14.1 build 2022-03-08T08:30:30.667857105+00:00
2022-03-12T13:47:52.410106942+06:00 INFO  shadowsocks socks TCP listening on 0.0.0.0:5634
2022-03-12T13:47:52.410254412+06:00 INFO  shadowsocks socks5 UDP listening on 0.0.0.0:5634
2022-03-12T13:49:04.986322022+06:00 ERROR received invalid UDP associate packet: b"\x01\x00\x00\x00e\xd9a\...

Are my configs incorrect? Or is there a way to fix it?

zonyitoo commented 2 years ago

The sslocal is running in socks5 mode, so it is of course reporting errors when you are trying to send WireGuard UDP packets to it.

database64128 commented 2 years ago

Use tunnel mode instead of SOCKS5.

airled commented 2 years ago

I add

"protocol": "tunnel"

to shadowsocks config, but now it forces me to specify forward-addr parameter. Can you please provide me what exactly should I use here to tunnel wireguard traffic?

zonyitoo commented 2 years ago

The forward address is the target address that specifies the "tunnel"'s endpoint. In this case, set forward address to your WireGuard server's address.

airled commented 2 years ago

Thank you for your help, but no luck. WireGuard is not working. No shadowsocks log either.

zonyitoo commented 2 years ago

What did you see? Try to add -v when starting sslocal and ssserver.

airled commented 2 years ago

./ssserver --server-addr '0.0.0.0:14000' --encrypt-method 'chacha20-ietf-poly1305' -k 'pass' -u -v

2022-03-12T09:52:20.779371491+01:00 INFO  [27707:139991275227520] [shadowsocks_rust::service::server] shadowsocks server 1.14.1 build 2022-03-08T08:30:30.667857105+00:00
2022-03-12T09:52:20.781948930+01:00 INFO  [27707:139991275227520] [shadowsocks_service::server::udprelay] shadowsocks udp server listening on 0.0.0.0:14000

./sslocal --local-addr '0.0.0.0:5634' --server-addr 'ip:14000' --encrypt-method 'chacha20-ietf-poly1305' -k 'pass' --protocol 'tunnel' -u -f 'ip' -v

2022-03-12T14:52:35.231772575+06:00 INFO  [8292:139683997137408] [shadowsocks_rust::service::local] shadowsocks local 1.14.1 build 2022-03-08T08:30:30.667857105+00:00
2022-03-12T14:52:35.233169122+06:00 INFO  [8292:139683997137408] [shadowsocks_service::local::tunnel::udprelay] shadowsocks UDP tunnel listening on 0.0.0.0:5634
2022-03-12T14:52:49.502851883+06:00 DEBUG [8292:139683997137408] [shadowsocks_service::local::tunnel::udprelay] created udp association for 127.0.0.1:52863
2022-03-12T14:52:49.567744566+06:00 DEBUG [8292:139683997132352] [shadowsocks::dns_resolver::resolver] auto-reload /etc/resolv.conf

That's all. No wireguard traffic.

zonyitoo commented 2 years ago 
2022-03-12T14:52:49.502851883+06:00 DEBUG [8292:139683997137408] [shadowsocks_service::local::tunnel::udprelay] created udp association for 127.0.0.1:52863

It had already created an association for your wireguard client. But the ssserver didn't receive anything, which is:

  1. --server-addr 'ip:14000' is not connectable from sslocal
  2. Something is blocking sslocal to make the actual request to the ssserver

The -f is ip:port, which is your Wireguard server's ip:port (address).

airled commented 2 years ago

./sslocal --local-addr '0.0.0.0:5634' --server-addr 'ip:14000' --encrypt-method 'chacha20-ietf-poly1305' -k 'pass' --protocol 'tunnel' -u -f 'ip:51820' -v

2022-03-12T15:05:58.296755016+06:00 INFO  [9719:140588295617024] [shadowsocks_rust::service::local] shadowsocks local 1.14.1 build 2022-03-08T08:30:30.667857105+00:00
2022-03-12T15:05:58.301582638+06:00 INFO  [9719:140588295617024] [shadowsocks_service::local::tunnel::udprelay] shadowsocks UDP tunnel listening on 0.0.0.0:5634
2022-03-12T15:06:03.503786903+06:00 DEBUG [9719:140588295617024] [shadowsocks_service::local::tunnel::udprelay] created udp association for 127.0.0.1:40035
2022-03-12T15:06:04.319058997+06:00 DEBUG [9719:140588293510720] [shadowsocks::dns_resolver::resolver] auto-reload /etc/resolv.conf

Still no traffic.

zonyitoo commented 2 years ago

You won't see any traffic from logs. Does ssserver show created udp association log?

airled commented 2 years ago

No, only initialization entries.

zonyitoo commented 2 years ago

So I think sslocal couldn't send any UDP packets to ip:14000, or ssserver couldn't receive any packets from sslocal.

You may try to run both sslocal and sserver locally in the same environment and add -vvv on both sslocal and ssserver and see what exactly happening.

database64128 commented 2 years ago

Note that the forward address is relative to the Shadowsocks server. That is, if you run your ssserver on the same machine as your WireGuard server, you can use -f [::1]:port.

airled commented 2 years ago

So I think sslocal couldn't send any UDP packets to ip:14000, or ssserver couldn't receive any packets from sslocal.

You may try to run both sslocal and sserver locally in the same environment and add -vvv on both sslocal and ssserver and see what exactly happening.

I run both ssserver and sslocal on my local machine. When I up wg interface I see association entries in both logs. Are you sure, that my run commands for ssserver and sslocal are correct? I'm pretty sure that udp traffic is not blocked since wireguard works absolutely fine without shadowsocks .

zonyitoo commented 2 years ago

If sslocal and ssserver are both running on the same machine, try to set —server-addr 127.0.0.1:14000.

I don’t have much ideas about why your ssserver didn’t receive any packets from sslocal.

airled commented 2 years ago

Well, thank you for trying to help. Seems like I should try to find another solution.

database64128 commented 2 years ago

Did you lower your WireGuard tunnel's MTU? The default 1420 assumes using IPv6 with a PMTU of 1500. When encapsulating WireGuard packets into Shadowsocks, the final Shadowsocks packet may exceed your on-path MTU and get silently dropped by routers.

You are using ChaCha20-Poly1305, which introduces overhead of a 32-byte salt and a 16-byte tag. The SOCKS address header should be 1+4+2 bytes long for IPv4 targets and 1+16+2 bytes long for IPv6 targets. Therefore, you should lower your WireGuard tunnel MTU to 1420-32-(1+16+2)-16=1353.

airled commented 2 years ago

Did you lower your WireGuard tunnel's MTU? The default 1420 assumes using IPv6 with a PMTU of 1500. When encapsulating WireGuard packets into Shadowsocks, the final Shadowsocks packet may exceed your on-path MTU and get silently dropped by routers.

You are using ChaCha20-Poly1305, which introduces overhead of a 32-byte salt and a 16-byte tag. The SOCKS address header should be 1+4+2 bytes long for IPv4 targets and 1+16+2 bytes long for IPv6 targets. Therefore, you should lower your WireGuard tunnel MTU to 1420-32-(1+16+2)-16=1353.

Thank you, I set wg MTU to 1200 but still no luck.

zonyitoo commented 2 years ago

I don't think there are any obvious obstacles to use shadowsocks' tunnel to relay WireGuard's UDP packets.

An idea come up to my mind: Maybe the packet sent from sslocal is routed back to WireGuard's tun interface, which is why it cannot reached to ssserver? I don't know. You could try to run the sslocal on another machine and see if it works.

database64128 commented 2 years ago

On another note, you should not set the tunnel MTU anywhere below 1280. 1280 is the minimum MTU of IPv6.

airled commented 2 years ago

@zonyitoo @database64128 Something changed. I disabled automatic wireguard routing with Table = off in wg config. Now ssserver see association but raise this error:

2022-03-12T18:24:42.853568343+01:00 INFO  [23061:139631187380608] [shadowsocks_rust::service::server] shadowsocks server 1.14.1 build 2022-03-08T08:30:30.667857105+00:00
2022-03-12T18:24:42.855348318+01:00 INFO  [23061:139631187380608] [shadowsocks_service::server::udprelay] shadowsocks udp server listening on 0.0.0.0:14000
2022-03-12T18:25:10.750847648+01:00 DEBUG [23061:139631187380608] [shadowsocks_service::server::udprelay] created udp association for 188.232.116.227:52075
2022-03-12T18:25:10.751594327+01:00 ERROR [23061:139631187380608] [shadowsocks_service::server::udprelay] udp server recv_from failed with error: invalid tag-in

Any ideas?

zonyitoo commented 2 years ago

@zonyitoo @database64128 Something changed. I disabled automatic wireguard routing with Table = off in wg config. Now ssserver see association but raise this error:

2022-03-12T18:24:42.853568343+01:00 INFO  [23061:139631187380608] [shadowsocks_rust::service::server] shadowsocks server 1.14.1 build 2022-03-08T08:30:30.667857105+00:00
2022-03-12T18:24:42.855348318+01:00 INFO  [23061:139631187380608] [shadowsocks_service::server::udprelay] shadowsocks udp server listening on 0.0.0.0:14000
2022-03-12T18:25:10.750847648+01:00 DEBUG [23061:139631187380608] [shadowsocks_service::server::udprelay] created udp association for 188.232.116.227:52075
2022-03-12T18:25:10.751594327+01:00 ERROR [23061:139631187380608] [shadowsocks_service::server::udprelay] udp server recv_from failed with error: invalid tag-in

Any ideas?

This is a crypto error, which means that the ssserver failed to decrypt the received packet.

Do sslocal and ssserver share the same method and password?

Or maybe it is packet too large to fit in an UDP frame? Did you set the maximum UDP packet size in WG?

airled commented 2 years ago

It works now without automatic wg routing. -f value was incorrect. It was as ip:14000 but should be ip:51820. I can now route my wg traffic via shadowsocks tunnel. But also I should investigate now how to forward all my traffic again via wg, not only custom routes.

Theo-Silfa commented 2 years ago

Hey guys. I'm trying to do almost the same thing here but it seems that ssserver forwards udp traffic only one direction. I do nc -u -l <tunneled-port> on the server and nc -u localhost <sslocal-local-port> on the client. Typing smth in to the client will show the data on the server but the reverse doesn't work. Tried the same thing with tcp and everything worked. Same behaviour is observed in libev implementation. Is it a bug or expected behaviour? 😕

zonyitoo commented 2 years ago

Just simply use dig xxxxxxx and you can verify whether the UDP association works as expected.

drew1kun commented 1 year ago

It works now without automatic wg routing. -f value was incorrect. It was as ip:14000 but should be ip:51820. I can now route my wg traffic via shadowsocks tunnel. But also I should investigate now how to forward all my traffic again via wg, not only custom routes.

Hi @airled, I am trying to do the same thing but without any luck. Could you please show client config and the command how do you run it? Thank you.

drew1kun commented 1 year ago

@airled My setup:

SS server config:

{
  "servers": [
    {
      "mode":"tcp_and_udp",
      "server":"0.0.0.0",
      "server_port":8388,
      "fast_open":false,
      "timeout":300,
      "method":"aes-256-gcm",
      "password":"PASS"
    }
  ]
}

SS Client Config ~/shadowsocks-rust-wg-tunnel.json (in accordance with documentation):

{
    "mode":"udp_only",
    "protocol":"tunnel",
    "server":"<ss_server_ip>",
    "server_port":8388,
    "local_address":"127.0.0.1",
    "local_port":51821,
    "password":"PASS",
    "method":"aes-256-gcm",
    "forward_address":"<wireguard_ip>",
    "forward_port":"<wireguard_port_51820>"
}

Running client with:

sslocal -c ~/shadowsocks-rust-wg-tunnel.json
config integrity check failed, missing `forward_addr` in configuration

What am I doing wrong? Should I remove "protocol":"tunnel", and use "tunnel_address": "127.0.0.1:51820" similar to shadowsocks-libev? Or should it be "tunnel_address": "<wireguard_ip>:51820" instead? Please help.

drew1kun commented 1 year ago

Well, I managed to make it work setting MTU = 1384 on client wireguard config and running ss client this way:

sslocal --local-addr '127.0.0.1:51821' --server-addr '<ss_server_IP>:8388' --encrypt-method 'aes-256-gcm' -k 'SS_PASS' --protocol 'tunnel' -u -f '<wireguard_server_IP>:51820' -v

But how do I translate this into config? With "protocol":"tunnel", set in ~/shadowsocks-rust-wg-tunnel.json still receiving the error mentioned above. Any ideas?

drew1kun commented 1 year ago

Posting here update for information purposes. The working config looks like this:

{
    "server":"<ss_server_IP>",
    "server_port":8388,
    "password":"SS_PASS",
    "method":"aes-256-gcm",
    "locals": [
        {
            "mode":"udp_only",
            "protocol":"tunnel",
            "forward_address":"<wireguard_IP>",
            "forward_port":"<wireguard_port_51820>",
            "local_address":"127.0.0.1",
            "local_port":51821
        }
    ]
}

WG config:

[Interface]
PrivateKey = PRIV_KEY
ListenPort = 51820
Address = 10.0.9.10/24
DNS = 1.1.1.1, search.domain
MTU = 1384

[Peer]
PublicKey = PUB_KEY
PresharedKey = PRESHARED_KEY
AllowedIPs = 0.0.0.0/0, ::/0
Endpoint = localhost:51821
PersistentKeepalive = 25

The only thing is that wireguard connection now looks somewhat unstable. I set up MTU = 1384 . But it seems it does not work sometimes. I run pings 8.8.8.8 and ping 192.168.10.10(host within the VPN network), in order to see if Wireguard over Shadowsocks works after I press connect on wireguard UI. Sometimes I need to disconnect and connect again, in order to start receiving the pings... It is kind of a lottery instead of stable connection. @database64128 could that be related to the MTU number I use, and how do I find out the recommended value? Maybe someone could suggest any other configuration in order to maximize stability of WG tunneling over SS? Thank you.

database64128 commented 1 year ago

@drew1kun aes-256-gcm has the same amount of overhead as chacha20-poly1305, so you can just follow the recommendation in https://github.com/shadowsocks/shadowsocks-rust/issues/785#issuecomment-1065854775.

You might also be interested in this project of mine: https://github.com/database64128/swgp-go. The zero-overhead mode can be used to proxy WireGuard traffic without affecting the tunnel MTU.

Bromelioideae commented 1 year ago

Posting here update for information purposes. The working config looks like this:

{
    "server":"<ss_server_IP>",
    "server_port":8388,
    "password":"SS_PASS",
    "method":"aes-256-gcm",
    "locals": [
        {
            "mode":"udp_only",
            "protocol":"tunnel",
            "forward_address":"<wireguard_IP>",
            "forward_port":"<wireguard_port_51820>",
            "local_address":"127.0.0.1",
            "local_port":51821
        }
    ]
}

WG config:

[Interface]
PrivateKey = PRIV_KEY
ListenPort = 51820
Address = 10.0.9.10/24
DNS = 1.1.1.1, search.domain
MTU = 1384

[Peer]
PublicKey = PUB_KEY
PresharedKey = PRESHARED_KEY
AllowedIPs = 0.0.0.0/0, ::/0
Endpoint = localhost:51821
PersistentKeepalive = 25

Thank you, thank you, thank you! I was finally able to get this config working after hours of tinkering. I initially got it to work a few hours ago but after restarting the server to confirm everything was okay, nothing was. My wireguard was slow, the regular shadowsocks connection was slow and the tunnel didn't work any at all. If I wasn't fully conscious I would have thought I dreamt it. Instead of going back to the basics I was searching for irrelevant solutions as I knew the connections were stable before and I had seen the tunnel in operation. The base issue turned out to be that both my unbound dns server and another instance of shadowsocks were both disabled, not quite sure how that happened but those were the causes of my slow regular ss and my regular wireguard,

After resolving those issues the regular connections became stable again so I went back to trying the tunnel. I am currently using Windows so I defaulted to using the Windows gui app. That works fine for ss and cloak+ss but each time I edited the config to add the locals tunnel section it would remove the section on closing the app. I went down the rabbit hole and had to follow the config from a commercial VPN vendor to realise that I had to use the command line app. So I downloaded the windows "msvc" version and tried that and it worked without issue .

Then I started having the issues mentioned before so after getting the connections stable and another long round of searching without touching the config that I knew to work I realised that I was trying to force cloak and another unobfuscated server to live on the same port 443. Despite configuring the cloaked server to only use UDP, I the ss client seemed to prefer to connect to the cloak server on TCP 443 when connecting even though a ss server on UDP 443 was available. As a test I disable the cloak server on TCP 443 and I am now using the tunnel.

I hope I will never have the real NEED to use this solution but I hope your work on this and my little clarifiers will assist those who do. My next steps are to try and get the tunnel obfuscated with cloak. In the meantime I need to figure how to run the command line app automatically on windows boot and how to create a shortcut or something that launches the app and puts it in the background while still available to shutdown.

hiiamboris commented 10 months ago

@drew1kun have you found a solution for the lottery behavior? For me it has like 5-10% chance to connect successfully.

drew1kun commented 10 months ago

@hiiamboris unfortunately, the only solution I have found so far is to use shadowsocks-libev instead.

hiiamboris commented 10 months ago

Which on Windows isn't an option because doesn't support tunnels or any custom config :/

zonyitoo commented 10 months ago

@hiiamboris unfortunately, the only solution I have found so far is to use shadowsocks-libev instead.

What's the actual problem you have found at that time exactly?