3proxy / 3proxy

3proxy - tiny free proxy server
Other
3.85k stars 760 forks source link

Proxy UDP behind AWS 1-1 NAT #681

Open dev-pingfighter opened 3 years ago

dev-pingfighter commented 3 years ago

Hey and much thanks for your work! @z3APA3A Hopefully this question will be helpful for everyone else who has the same problem.

So I am trying to set up a socks proxy for tcp and udp connections (via UDP ASSOCIATE) and while it's working fine on DigitalOcean, it's not working on AWS. Basically because when an EC2 instance has a public IPv4 address associated with it (no matter whether it's a public IP from the AWS pool or Elastic IP), it can't bind a socket to the public IP address, because it's statically NAT-ed to the private IP by the Internet Gateway -- the instance itself is not aware of the public IP address (output of ip addr show command has no public IP in it). If socks proxy is bound to the private IP or 0.0.0.0 then tcp works just fine, but reported IP for the UDP ASSOCIATE request is a private IP, which means, that sent datagrams from the client are silently dropped and aren't received by the proxy server. I have tried to use -N flag with the next config:

socks -N18.157.80.150 -e172.31.11.72 -i172.31.11.72 -p1091

but unfortunately in this case socks proxy returns 00012 code error, which means that socks proxy "failed to bind()" according to this doc https://3proxy.ru/howtoe.asp#ERRORS

It's fair enough! So I have tried to test it by adding secondary ip to the network interface with sudo ip addr add 18.157.80.150/32 dev eth0. In this case socks proxy returned no error, but the port has been assigned to an outbound connection. Here are the logs I have so far:

  1. I am using dante-client socksify to send datagrams:

    socksify nc -v -u 3.65.20.100 5000
    Sep 22 22:33:45 (1632339225.441829) nc[7157]: info: Dante/client v1.4.2 running
    Sep 22 22:33:45 (1632339225.443525) nc[7157]: info: Rconnect(): fd 3, address 3.65.20.100.5000
    Sep 22 22:33:45 (1632339225.443679) nc[7157]: info: socks_requestpolish(): searching for direct route for udpassociate, protocol udp, src 0.0.0.0.0, dst 3.65.20.100.5000, authmethod -1
    Sep 22 22:33:45 (1632339225.444090) nc[7157]: info: socks_connectroute(): have socks_v5 route (route #1) to 3.65.20.100.5000 via 18.157.80.150.1091
    Sep 22 22:33:45 (1632339225.444144) nc[7157]: info: socks_connecthost(): connect to 18.157.80.150.1091 on <N/A> side from 0.0.0.0.0, fd 4.  Timeout is -1
    Sep 22 22:33:45 (1632339225.488056) nc[7157]: info: socks_connecthost(): connect to 18.157.80.150.1091 from 192.168.0.103.38894 on fd 4 ok (no system error)
    Sep 22 22:33:45 (1632339225.488165) nc[7157]: info: socks_negotiate(): initiating socks_v5 negotiation with control-fd 4 (laddr: 192.168.0.103.38894, raddr: 18.157.80.150.1091, protocol: tcp), data-fd 3 (laddr: 192.168.0.103.43578, raddr: N/A, protocol: udp), req.host = 192.168.0.103.43578
    Sep 22 22:33:45 (1632339225.488189) nc[7157]: info: negotiate_method(): offering proxy server #1 method: username
    Sep 22 22:33:45 (1632339225.530188) nc[7157]: info: negotiate_method(): proxy server selected method username
    Sep 22 22:33:45 (1632339225.530294) nc[7157]: info: socks_getusername(): using socks username from environment: "RandomName"
    Sep 22 22:33:45 (1632339225.530307) nc[7157]: info: clientmethod_uname(): offering username "RandomName", password ******** to server
    Sep 22 22:33:45 (1632339225.573310) nc[7157]: info: clientmethod_uname(): received server response: 0x1, 0x0
    Sep 22 22:33:45 (1632339225.573358) nc[7157]: info: negotiate_method(): established v5 connection using method 2
    Sep 22 22:33:45 (1632339225.573380) nc[7157]: info: socks_sendrequest(): sending request to server: VER: 5 CMD: 3 FLAG: 0 ATYP: 1 address: 192.168.0.103.43578
    Sep 22 22:33:45 (1632339225.615370) nc[7157]: info: socks_recvresponse(): received response from server: VER: 5 REP: 0 FLAG: 0 ATYP: 1 address: 172.31.11.72.55948
    Sep 22 22:33:45 (1632339225.615407) nc[7157]: info: serverreplyisok(): version 5, command 3, reply 0
    Sep 22 22:33:45 (1632339225.615457) nc[7157]: info: udpsetup(): connecting fd 3 from 192.168.0.103.43578 to socks_v5-server 172.31.11.72.55948 succeeded: no system error
    Sep 22 22:33:45 (1632339225.615484) nc[7157]: info: Rconnect(): route set up for fd 3 to 3.65.20.100.5000 is a socks_v5 route
    Connection to 3.65.20.100 5000 port [udp/rplay] succeeded!
  2. At the same time on the server where socks proxy is installed:

    
    sudo netstat -utlpn | grep 3proxy

tcp 0 0 172.31.11.72:1091 0.0.0.0: LISTEN 79357/3proxy
udp 0 0 18.157.80.150:43875 0.0.0.0:
79357/3proxy
udp 0 0 172.31.11.72:55948 0.0.0.0:* 79357/3proxy

3. 3proxy logs I have after terminating tcp connection (I am using JSON format for easier parsing):
```bash
logformat "-\""+_G{""time_unix"":%t, ""proxy"":{""type:"":""%N"", ""port"":%p}, ""error"":{""code"":""%E""}, ""auth"":{""user"":""%U""}, ""client"":{""ip"":""%C"", ""port"":%c}, ""server"":{""ip"":""%R"", ""port"":%r}, ""bytes"":{""sent"":%O, ""received"":%I}, ""request"":{""hostname"":""%n""}, ""message"":""%T""}"
{"time_unix":1632339998, "proxy":{"type:":"SOCK5", "port":1091}, "error":{"code":"00000"}, "auth":{"user":"RandomName"}, "client":{"ip":"my ISP public IP here", "port":38894}, "server":{"ip":"192.168.0.103", "port":43578}, "bytes":{"sent":0, "received":0}, "request":{"hostname":"192.168.0.103"}, "message":"UDPMAP 192.168.0.103:43578"}

So as you can see there are no errors, but the bound port of -N IP is the opposite of the one I need and reported IP to the UDP ASSOCIATE command is the private IP.

Please let me know if I can somehow achieve what I need and if there are any more required data from my side, then I will gladly provide it. Thanks!

z3APA3A commented 2 years ago

-N should work in your case, it should not lead to bind() errors, because -N address is not used for binding, this address is reported to client. Can you provide more info/log examples?

dev-pingfighter commented 2 years ago

@z3APA3A, thanks for your reply!

Unfortunately it's not working. I have just checked it, here are the logs:

  1. Socksify logs
socksify nc -v -u 3.65.20.100 5000
Sep 27 11:21:20 (1632730880.167427) nc[7661]: info: Dante/client v1.4.2 running
Sep 27 11:21:20 (1632730880.168652) nc[7661]: info: Rconnect(): fd 3, address 3.65.20.100.5000
Sep 27 11:21:20 (1632730880.168785) nc[7661]: info: socks_requestpolish(): searching for direct route for udpassociate, protocol udp, src 0.0.0.0.0, dst 3.65.20.100.5000, authmethod -1
Sep 27 11:21:20 (1632730880.169003) nc[7661]: info: socks_connectroute(): have socks_v5 route (route #1) to 3.65.20.100.5000 via 3.70.44.170.1091
Sep 27 11:21:20 (1632730880.169036) nc[7661]: info: socks_connecthost(): connect to 3.70.44.170.1091 on <N/A> side from 0.0.0.0.0, fd 4.  Timeout is -1
Sep 27 11:21:20 (1632730880.211342) nc[7661]: info: socks_connecthost(): connect to 3.70.44.170.1091 from 192.168.0.103.57942 on fd 4 ok (no system error)
Sep 27 11:21:20 (1632730880.211450) nc[7661]: info: socks_negotiate(): initiating socks_v5 negotiation with control-fd 4 (laddr: 192.168.0.103.57942, raddr: 3.70.44.170.1091, protocol: tcp), data-fd 3 (laddr: 192.168.0.103.42342, raddr: N/A, protocol: udp), req.host = 192.168.0.103.42342
Sep 27 11:21:20 (1632730880.211482) nc[7661]: info: negotiate_method(): offering proxy server #2 methods: none, username
Sep 27 11:21:20 (1632730880.251678) nc[7661]: info: negotiate_method(): proxy server selected method username
Sep 27 11:21:20 (1632730880.251721) nc[7661]: info: socks_getusername(): using socks username from environment: "Test"
Sep 27 11:21:20 (1632730880.251734) nc[7661]: info: clientmethod_uname(): offering username "Test", password ******** to server
Sep 27 11:21:20 (1632730880.292351) nc[7661]: info: clientmethod_uname(): received server response: 0x1, 0x0
Sep 27 11:21:20 (1632730880.292389) nc[7661]: info: negotiate_method(): established v5 connection using method 2
Sep 27 11:21:20 (1632730880.292408) nc[7661]: info: socks_sendrequest(): sending request to server: VER: 5 CMD: 3 FLAG: 0 ATYP: 1 address: 192.168.0.103.42342
Sep 27 11:21:50 (1632730910.391959) nc[7661]: info: socks_blacklist(): blacklisting route #1.  Reason: premature eof from proxy server while waiting for response
Sep 27 11:21:50 (1632730910.392144) nc[7661]: warning: udpsetup(): socks_negotiate() failed: premature eof from proxy server while waiting for response
Sep 27 11:21:50 (1632730910.392176) nc[7661]: info: Rconnect(): udpsetup() returned no route to 3.65.20.100.5000 for fd 3: premature eof from proxy server while waiting for response
nc: connect to 3.65.20.100 port 5000 (udp) failed: Network is unreachable
  1. 3proxy config:
    
    #!/bin/3proxy
    config /etc/3proxy/3proxy.cfg

Configure nservers to avoid unsafe gethostbyname() usage

nserver 1.0.0.1 nserver 1.1.1.1 nserver 8.8.4.4 nserver 8.8.8.8

Set nscache to save speed, traffic and bandwidth

nscache 65536

Default timeout values

timeouts 1 5 30 60 180 1800 15 60 15 5

Logging docs: https://3proxy.org/doc/howtor.html#LOGFORMAT

By default logs to STDOUT

log

Logs in JSON format

logformat "-\""+_G{""time_unix"":%t, ""proxy"":{""type:"":""%N"", ""port"":%p}, ""error"":{""code"":""%E""}, ""auth"":{""user"":""%U""}, ""client"":{""ip"":""%C"", ""port"":%c}, ""server"":{""ip"":""%Q"", ""port"":%q}, ""bytes"":{""sent"":%O, ""received"":%I}, ""request"":{""hostname"":""%n""}, ""message"":""%T""}"

maxconn 1024

AUTH_SETTINGS

socks -N3.70.44.170 -e172.31.11.72 -i172.31.11.72 -p1091

flush


3. 3proxy logs

```bash
{"time_unix":1632730852, "proxy":{"type:":"SOCKS", "port":1091}, "error":{"code":"00000"}, "auth":{"user":"-"}, "client":{"ip":"172.31.11.72", "port":1091}, "server":{"ip":"0.0.0.0", "port":0}, "bytes":{"sent":0, "received":0}, "request":{"hostname":"[0.0.0.0]"}, "message":"Accepting connections [1/1686107904]"}
{"time_unix":1632730910, "proxy":{"type:":"SOCK5", "port":1091}, "error":{"code":"00012"}, "auth":{"user":"Test"}, "client":{"ip":"My ISP IP here", "port":57942}, "server":{"ip":"192.168.0.103", "port":42342}, "bytes":{"sent":0, "received":0}, "request":{"hostname":"192.168.0.103"}, "message":"UDPMAP 192.168.0.103:42342"}

3proxy version is 0.9.4