search

fatedier / frp

A fast reverse proxy to help you expose a local server behind a NAT or firewall to the internet.
Apache License 2.0
80.49k stars 12.74k forks source link

IPv6 proxy weirdness #4180

Open dharapvj opened 3 weeks ago

dharapvj commented 3 weeks ago

Bug Description

If frpc connects to frps server via IPv6 - I cannot connect to local services exposed via container e.g. nginx docker container. But same frpc connects to frps via IPv4, same local service can get connected.

More details can be found in the reproduction steps.

frpc Version

0.57.0

frps Version

0.57.0

System Architecture

linux/amd (client) linux/arm64 (server)

Configurations

########  frps.toml - server_ipv6
bindAddr = "::"
bindPort = 2333
log.level = "debug"

########  frps.ini - server_ipv4
[common]
bind_port = 2333

######### frpc.toml
serverAddr = "IPv6_IP_ADDR_OF_SERVER_IPv6"
serverPort = 2333
log.level = "trace"

[[proxies]]
name = "ssh"
type = "tcp"
localIp = "127.0.0.1"
localPort = 22
remotePort = 43332

[[proxies]]
name = "web"
type = "tcp"
localIp = "127.0.0.1"
localPort = 9000

Logs

Logs:

❯ frpc -c /etc/frp/frpc.toml
2024-04-25 00:10:33.130 [I] [sub/root.go:142] start frpc service for config file [/etc/frp/frpc.toml]
2024-04-25 00:10:33.130 [I] [client/service.go:294] try to connect to server...
2024-04-25 00:10:33.154 [I] [client/service.go:286] [3dede59ef6706e74] login to server success, get run id [3dede59ef6706e74]
2024-04-25 00:10:33.154 [I] [proxy/proxy_manager.go:173] [3dede59ef6706e74] proxy added: [ssh web]
2024-04-25 00:10:33.154 [T] [proxy/proxy_wrapper.go:200] [3dede59ef6706e74] [ssh] change status from [new] to [wait start]
2024-04-25 00:10:33.154 [T] [proxy/proxy_wrapper.go:200] [3dede59ef6706e74] [web] change status from [new] to [wait start]
2024-04-25 00:10:33.154 [D] [client/control.go:244] [3dede59ef6706e74] send heartbeat to server
2024-04-25 00:10:33.160 [I] [client/control.go:170] [3dede59ef6706e74] [ssh] start proxy success
2024-04-25 00:10:33.165 [W] [client/control.go:168] [3dede59ef6706e74] [web] start error: port unavailable
2024-04-25 00:10:33.165 [D] [client/control.go:195] [3dede59ef6706e74] receive heartbeat from server
2024-04-25 00:11:03.155 [D] [client/control.go:244] [3dede59ef6706e74] send heartbeat to server
2024-04-25 00:11:03.160 [D] [client/control.go:195] [3dede59ef6706e74] receive heartbeat from server
2024-04-25 00:11:03.172 [T] [proxy/proxy_wrapper.go:200] [3dede59ef6706e74] [web] change status from [start error] to [wait start]
2024-04-25 00:11:03.177 [W] [client/control.go:168] [3dede59ef6706e74] [web] start error: port unavailable

Logs when connected to IPv4 Server:

frpc[192002]: WARNING: ini format is deprecated and the support will be removed in the future, please use yaml/json/toml format instead!
frpc[192002]: 2024-04-25 00:19:30.606 [I] [sub/root.go:142] start frpc service for config file [/etc/frp/frpc.ini]
frpc[192002]: 2024-04-25 00:19:30.607 [I] [client/service.go:294] try to connect to server...
frpc[192002]: 2024-04-25 00:19:30.633 [I] [client/service.go:286] [055be5915b8650e0] login to server success, get run id [055be5915b8650e0]
frpc[192002]: 2024-04-25 00:19:30.634 [I] [proxy/proxy_manager.go:173] [055be5915b8650e0] proxy added: [ssh tcp_port_0 tcp_port_1]
frpc[192002]: 2024-04-25 00:19:30.634 [T] [proxy/proxy_wrapper.go:200] [055be5915b8650e0] [ssh] change status from [new] to [wait start]
frpc[192002]: 2024-04-25 00:19:30.634 [T] [proxy/proxy_wrapper.go:200] [055be5915b8650e0] [tcp_port_1] change status from [new] to [wait start]
frpc[192002]: 2024-04-25 00:19:30.634 [T] [proxy/proxy_wrapper.go:200] [055be5915b8650e0] [tcp_port_0] change status from [new] to [wait start]
frpc[192002]: 2024-04-25 00:19:30.635 [D] [client/control.go:244] [055be5915b8650e0] send heartbeat to server
frpc[192002]: 2024-04-25 00:19:30.642 [I] [client/control.go:170] [055be5915b8650e0] [ssh] start proxy success
frpc[192002]: 2024-04-25 00:19:30.642 [I] [client/control.go:170] [055be5915b8650e0] [tcp_port_1] start proxy success
frpc[192002]: 2024-04-25 00:19:30.642 [I] [client/control.go:170] [055be5915b8650e0] [tcp_port_0] start proxy success
frpc[192002]: 2024-04-25 00:19:30.646 [D] [client/control.go:195] [055be5915b8650e0] receive heartbeat from server
frpc[192002]: 2024-04-25 00:19:47.380 [D] [proxy/proxy_wrapper.go:260] [055be5915b8650e0] [tcp_port_1] start a new work connection, localAddr: 192.168.1.52:34486 remoteAddr: 13.235.216.249:2333

Steps to reproduce

  1. I have frps running on a IPv6 enabled server (server_ipv6) with minimal config as below in AWS
  2. I have another frps running on another IPv4 enabled server (server_ipv4) with same config in AWS
  3. I have frpc running in my local machine.
  4. I am trying to proxy a web-server running in my local running inside a container with ports published to host. e.g. podman run -p 9000:9000 -p 9001:9001 quay.io/minio/minio server /data --console-address ":9001" But anything else like nginx will also do. I can access service via curl -v localhost:9000 on the local machine host.
  5. If I connect my frpc to IPv4 server's frps, everything work fine.
  6. If I connect my frpc to IPv6 server's frps, I get port unavailable error.

Affected area

fatedier commented 3 weeks ago

2024-04-25 00:11:03.177 [W] [client/control.go:168] [3dede59ef6706e74] [web] start error: port unavailable

You are not using the same configuration?

You need to check the error messages in the frps log.

dharapvj commented 3 weeks ago

I think it was late in night and I pasted wrong logs.. I recreated the scenario again. I also removed ssh proxy so that logs can be simple to look at.

Here is config: ipv4 server

[common]
bind_port = 2333

ipv6 server

bindAddr = "::"
bindPort = 2333
log.level = "debug"

local client config

serverAddr = "<IPv6_address>"
# serverAddr = "<IPv4_Address>"
serverPort = 2333
log.level = "trace"

[[proxies]]
name = "web"
type = "tcp"
localIp = "127.0.0.1"
localPort = 80
remotePort = 80

Here are logs: Connecting to ipv4 machine:

❯ frpc -c /etc/frp/frpc.toml
2024-04-25 22:37:43.498 [I] [sub/root.go:142] start frpc service for config file [/etc/frp/frpc.toml]
2024-04-25 22:37:43.498 [I] [client/service.go:294] try to connect to server...
2024-04-25 22:37:43.521 [I] [client/service.go:286] [a280a2e63b59f284] login to server success, get run id [a280a2e63b59f284]
2024-04-25 22:37:43.521 [I] [proxy/proxy_manager.go:173] [a280a2e63b59f284] proxy added: [web]
2024-04-25 22:37:43.521 [T] [proxy/proxy_wrapper.go:200] [a280a2e63b59f284] [web] change status from [new] to [wait start]
2024-04-25 22:37:43.521 [D] [client/control.go:244] [a280a2e63b59f284] send heartbeat to server
2024-04-25 22:37:43.527 [I] [client/control.go:170] [a280a2e63b59f284] [web] start proxy success
2024-04-25 22:37:43.527 [D] [client/control.go:195] [a280a2e63b59f284] receive heartbeat from server
^C

Connecting to ipv6 machine

❯ frpc -c /etc/frp/frpc.toml
2024-04-25 23:00:17.651 [I] [sub/root.go:142] start frpc service for config file [/etc/frp/frpc.toml]
2024-04-25 23:00:17.651 [I] [client/service.go:294] try to connect to server...
2024-04-25 23:00:17.684 [I] [client/service.go:286] [791c87b2fe506ae9] login to server success, get run id [791c87b2fe506ae9]
2024-04-25 23:00:17.684 [I] [proxy/proxy_manager.go:173] [791c87b2fe506ae9] proxy added: [web]
2024-04-25 23:00:17.684 [T] [proxy/proxy_wrapper.go:200] [791c87b2fe506ae9] [web] change status from [new] to [wait start]
2024-04-25 23:00:17.685 [D] [client/control.go:244] [791c87b2fe506ae9] send heartbeat to server
2024-04-25 23:00:17.690 [W] [client/control.go:168] [791c87b2fe506ae9] [web] start error: port unavailable
2024-04-25 23:00:17.690 [D] [client/control.go:195] [791c87b2fe506ae9] receive heartbeat from server
^C

Corresponding ipv6 server logs:

2024-04-25 17:30:13.847 [I] [frps/root.go:105] frps uses config file: frps.toml
2024-04-25 17:30:14.872 [I] [server/service.go:237] frps tcp listen on [::]:2333
2024-04-25 17:30:14.872 [I] [frps/root.go:114] frps started successfully
2024-04-25 17:30:17.680 [I] [server/service.go:575] [791c87b2fe506ae9] client login info: ip [[2402:b00:e:f009:1059:f37a:5f71:155f]:60050] version [0.57.0] hostname [] os [linux] arch [amd64]
2024-04-25 17:30:17.686 [W] [server/control.go:396] [791c87b2fe506ae9] new proxy [web] type [tcp] error: port unavailable
2024-04-25 17:30:17.687 [D] [server/control.go:432] [791c87b2fe506ae9] receive heartbeat
2024-04-25 17:30:17.687 [D] [server/control.go:243] [791c87b2fe506ae9] new work connection registered
2024-04-25 17:30:21.480 [D] [server/service.go:524] Accept new mux stream error: EOF
2024-04-25 17:30:21.480 [I] [server/control.go:359] [791c87b2fe506ae9] client exit success

The service is available locally

❯ curl -I 127.0.0.1
HTTP/1.1 200 OK
Server: openresty
Date: Thu, 25 Apr 2024 17:10:36 GMT
Content-Type: text/html
Content-Length: 1033
Last-Modified: Wed, 28 Feb 2024 22:25:56 GMT
Connection: keep-alive
ETag: "65dfb2f4-409"
Accept-Ranges: bytes
xqzr commented 3 weeks ago

看起来 frps 所在的服务器，80 端口被其他进程占用 使用 ss -tlnp sport 80 找到占用 80 端口的进程

xqzr commented 3 weeks ago

另外

bindAddr = "::"

不是必须的

fatedier commented 3 weeks ago

@dharapvj You can refer to the comments above to confirm if port 80 has been occupied and try removing bindAddr.

dharapvj commented 2 weeks ago
  1. port 80 is of course occupied on the client (frpc) machine. That's the port on which I want incoming traffic from the frps machine to get routed at. But please note the fact that if I change the server address in frpc config to an IPv4 adddress instead of an IPv6 address and everything works fine. Then frpc does not complain about port not being available!
  2. if I remove bindAddr - it tries to by default connect on IPv4 0.0.0.0. But my target server machine is in IPv6-only network where IPv4 traffic cannot reach.

I just request you to try out such a scenario once. I have given detailed information and logs so that it could be easier for you to try and reproduce. Can you please try reproducing the issue once?

fatedier commented 2 weeks ago

You cannot expect everyone to have the same environment as you do, there are tests related to IPv6 in end-to-end testing, and obvious errors will be directly detected.

If possible, please conduct more tests, try more ports or other configuration adjustments. Otherwise, I won't be able to provide much help.