版本0.58.0的客户端SSH频发性断连

islercn commented 5 months ago

Bug Description

frps部署在阿里云，frpc部署在多台内网Ubuntu22.04主机，服务端和客户端都是0.58.0版。自升级到0.58.0版本后，在外网ssh内网机器很快就会断连，最长应该不会超过5分钟，这时如果再连，还能连上，但很快又自动断连了。尝试服务器维持0.58.0版本不变，仅将客户端降为0.57.0版本后问题消失。

frpc Version

0.58.0

frps Version

0.58.0

System Architecture

ubuntu 22.04

Configurations

服务器设置 bindAddr = "0.0.0.0" bindPort = 10000 auth.method = "token" auth.token = "Pass5678"

客户端设置 serverAddr = "服务器IP" serverPort = 10000 transport.tls.enable = true auth.method = "token" auth.token = "Pass5678"

[[proxies]] name = "ssh-001" type = "tcp" localIP = "127.0.0.1" localPort = 2222 remotePort = 10001

Logs

2024-05-17 14:42:30.781 [I] [sub/root.go:142] start frpc service for config file [/home/xxx/frpc.toml] 2024-05-17 14:42:30.781 [I] [client/service.go:294] try to connect to server... 2024-05-17 14:42:30.802 [I] [client/service.go:286] [240bfa9ecca5d40f] login to server success, get run id [240bfa9ecca5d40f] 2024-05-17 14:42:30.802 [I] [proxy/proxy_manager.go:173] [240bfa9ecca5d40f] proxy added: [ssh-001] 2024-05-17 14:42:30.802 [T] [proxy/proxy_wrapper.go:200] [240bfa9ecca5d40f] [ssh-001] change status from [new] to [wait start] 2024-05-17 14:42:30.808 [I] [client/control.go:168] [240bfa9ecca5d40f] [ssh-001] start proxy success 2024-05-17 14:42:33.599 [D] [proxy/proxy_wrapper.go:260] [240bfa9ecca5d40f] [ssh-001] start a new work connection, localAddr: localip:53416 remoteAddr: serverip:10000 2024-05-17 14:42:33.599 [T] [proxy/proxy.go:144] [240bfa9ecca5d40f] [ssh-001] handle tcp work connection, useEncryption: false, useCompression: false 2024-05-17 14:42:33.599 [D] [proxy/proxy.go:210] [240bfa9ecca5d40f] [ssh-001] join connections, localConn(l[127.0.0.1:44216] r[127.0.0.1:2222]) workConn(l[localip:53416] r[serverip:10000]) 2024-05-17 14:42:56.595 [D] [proxy/proxy_wrapper.go:260] [240bfa9ecca5d40f] [ssh-001] start a new work connection, localAddr: localip:53416 remoteAddr: serverip:10000 2024-05-17 14:42:56.595 [T] [proxy/proxy.go:144] [240bfa9ecca5d40f] [ssh-001] handle tcp work connection, useEncryption: false, useCompression: false 2024-05-17 14:42:56.595 [D] [proxy/proxy.go:210] [240bfa9ecca5d40f] [ssh-001] join connections, localConn(l[127.0.0.1:44928] r[127.0.0.1:2222]) workConn(l[localip:53416] r[serverip:10000]) 2024-05-17 14:42:59.044 [D] [proxy/proxy.go:222] [240bfa9ecca5d40f] [ssh-001] join connections closed 2024-05-17 14:42:59.044 [T] [proxy/proxy.go:224] [240bfa9ecca5d40f] [ssh-001] join connections errors: [writeto tcp 127.0.0.1:44928->127.0.0.1:2222: read tcp 127.0.0.1:44928->127.0.0.1:2222: use of closed network connection] 2024-05-17 14:44:00.858 [T] [client/control.go:145] [240bfa9ecca5d40f] work connection closed before response StartWorkConn message: EOF 2024-05-17 14:44:00.858 [I] [client/service.go:294] [240bfa9ecca5d40f] try to connect to server... 2024-05-17 14:44:00.858 [D] [proxy/proxy.go:222] [240bfa9ecca5d40f] [ssh-001] join connections closed 2024-05-17 14:44:00.858 [T] [proxy/proxy.go:224] [240bfa9ecca5d40f] [ssh-001] join connections errors: [writeto tcp 127.0.0.1:44216->127.0.0.1:2222: read tcp 127.0.0.1:44216->127.0.0.1:2222: use of closed network connection] 2024-05-17 14:44:00.880 [I] [client/service.go:286] [240bfa9ecca5d40f] login to server success, get run id [240bfa9ecca5d40f] 2024-05-17 14:44:00.880 [I] [proxy/proxy_manager.go:173] [240bfa9ecca5d40f] proxy added: [ssh-001] 2024-05-17 14:44:00.880 [T] [proxy/proxy_wrapper.go:200] [240bfa9ecca5d40f] [ssh-001] change status from [new] to [wait start] 2024-05-17 14:44:00.886 [I] [client/control.go:168] [240bfa9ecca5d40f] [ssh-001] start proxy success 2024-05-17 14:44:15.044 [D] [proxy/proxy_wrapper.go:260] [240bfa9ecca5d40f] [ssh-001] start a new work connection, localAddr: localip:33750 remoteAddr: serverip:10000 2024-05-17 14:44:15.044 [T] [proxy/proxy.go:144] [240bfa9ecca5d40f] [ssh-001] handle tcp work connection, useEncryption: false, useCompression: false 2024-05-17 14:44:15.044 [D] [proxy/proxy.go:210] [240bfa9ecca5d40f] [ssh-001] join connections, localConn(l[127.0.0.1:42170] r[127.0.0.1:2222]) workConn(l[localip:33750] r[serverip:10000]) 2024-05-17 14:44:18.990 [D] [proxy/proxy_wrapper.go:260] [240bfa9ecca5d40f] [ssh-001] start a new work connection, localAddr: localip:33750 remoteAddr: serverip:10000 2024-05-17 14:44:18.990 [T] [proxy/proxy.go:144] [240bfa9ecca5d40f] [ssh-001] handle tcp work connection, useEncryption: false, useCompression: false 2024-05-17 14:44:18.990 [D] [proxy/proxy.go:210] [240bfa9ecca5d40f] [ssh-001] join connections, localConn(l[127.0.0.1:33576] r[127.0.0.1:2222]) workConn(l[localip:33750] r[serverip:10000]) 2024-05-17 14:44:19.054 [D] [proxy/proxy.go:222] [240bfa9ecca5d40f] [ssh-001] join connections closed 2024-05-17 14:44:19.054 [T] [proxy/proxy.go:224] [240bfa9ecca5d40f] [ssh-001] join connections errors: [writeto tcp 127.0.0.1:42170->127.0.0.1:2222: read tcp 127.0.0.1:42170->127.0.0.1:2222: use of closed network connection] 2024-05-17 14:44:21.191 [D] [proxy/proxy.go:222] [240bfa9ecca5d40f] [ssh-001] join connections closed 2024-05-17 14:44:21.191 [T] [proxy/proxy.go:224] [240bfa9ecca5d40f] [ssh-001] join connections errors: [writeto tcp 127.0.0.1:33576->127.0.0.1:2222: read tcp 127.0.0.1:33576->127.0.0.1:2222: use of closed network connection]

Steps to reproduce

服务器客户端升级至0.58.0，配置好后运行
通过frp的线路，使用ssh连接客户端
大约1-3分钟后ssh自动断开

Affected area

[ ] Docs
[ ] Installation
[ ] Performance and Scalability
[ ] Security
[X] User Experience
[ ] Test and Release
[ ] Developer Infrastructure
[ ] Client Plugin
[ ] Server Plugin
[ ] Extensions
[ ] Others

swanbylei commented 5 months ago

遇到同样的问题。frps服务端0.58.0，frpc为Windows版0.58.0。连接间歇性断开连接。使用旧版本的frpc，0.53.2版无任何问题。

MrLiuGangQiang commented 5 months ago

同样的问题我还以为自己没配好换成0.57.0就稳如老狗0.58.0就马上断

fatedier commented 5 months ago

请尝试分别单独做如下配置修改来进行验证测试：

frpc 和 frps 连接之后，不进行任何请求和访问，持续观察一段时间两端是否有连接断开的情况。
ssh 启用 keepalive，例如 ServerAliveInterval 60 之类的配置。
frpc 增加 transport.tcpMuxKeepaliveInterval = 30 的配置。
frpc 增加 transport.heartbeatInterval = 30 的配置。

xingtongsf commented 5 months ago

我的问题不知道是不是相同的。我的客户端使用的是tiny-frpc.因为tinyfrpc使用的是ssh连接服务器，每次能正常代理20秒左右吧，直接断线了。再重启又能代理20秒左右。服务器端位0.58,客户端没有配置transport.tcpMuxKeepaliveInterval = 30之类的选项

fatedier commented 5 months ago

@xingtongsf 你的问题请提交到 tiny-frpc 的 repo。

YuxuanZuo commented 4 months ago

请尝试分别单独做如下配置修改来进行验证测试：

frpc 和 frps 连接之后，不进行任何请求和访问，持续观察一段时间两端是否有连接断开的情况。

ssh 启用 keepalive，例如 ServerAliveInterval 60 之类的配置。

frpc 增加 transport.tcpMuxKeepaliveInterval = 30 的配置。

frpc 增加 transport.heartbeatInterval = 30 的配置。

这里有差不多的问题，代理的是TCP协议的Minecraft Java版，客户端和服务端版本均为0.58.0，frpc增加配置transport.heartbeatInterval = 30后问题修复

fatedier commented 4 months ago

@YuxuanZuo 其他的配置修改也请一并测试一下，方便定位具体的原因。

YuxuanZuo commented 4 months ago

@YuxuanZuo 其他的配置修改也请一并测试一下，方便定位具体的原因。

单独添加 transport.tcpMuxKeepaliveInterval = 30配置没有效果，单独添加transport.heartbeatInterval = 10或二者都可以修复该问题。根据我对服务端trace日志的观察，大概每隔十几秒钟就会有heartbeat timeout日志，应该是客户端没有按时发送心跳包

fatedier commented 4 months ago

@YuxuanZuo 完整的配置可以贴一下吗？release notes 里有说明这种情况应该只发生在 frps 是旧版本的情况下，确认服务端已经更新到了最新版本，我本地无法复现。

fatedier commented 4 months ago

@YuxuanZuo 如果使用旧的 INI 格式的话，默认值可能会有问题，但是看你的描述你应该用的不是 INI ？你这个可能是单独的问题，可以另外提交 issue 跟进，INI 的问题我会修复掉。

这个 issue 没人继续反馈 ssh 的问题的话我会关闭掉。

YuxuanZuo commented 4 months ago

@YuxuanZuo 如果使用旧的 INI 格式的话，默认值可能会有问题，但是看你的描述你应该用的不是 INI ？你这个可能是单独的问题，可以另外提交 issue 跟进，INI 的问题我会修复掉。

这个 issue 没人继续反馈 ssh 的问题的话我会关闭掉。

确认了一下，frps确实用的ini配置，经修改后全部恢复正常，感谢作者大大耐心解答！

islercn commented 4 months ago

前一阵是网络不好，屋里温度比较高时，丢包率能干到3%，最近调整了下网络，很少再出现丢包了，这个问题也没再出现过。。所以感觉是丢包导致的心跳包丢失？所以能否自适应调整心跳包频率，或者增加重传机制？

fatedier commented 4 months ago

@islercn 可以自行尝试 kcp 或者 quic。

MaxKingPor commented 4 months ago

0.58.1 client 端也会出现 try to connect to server 日志一分钟一次服务端添加
```
[transport]
tcpKeepalive = 10
```
之后就没出现过(测试阶段没出现过大概2个小时)

fatedier / frp