[讨论]性能怎么样,功能方面和实现方面如何设计的。

[qeatzy]

请问，性能怎样? 1 延时低吗，尤其是打开多个浏览器窗口时。（比如用Vimium，谷歌搜索后一次打开10来个后台窗口，然后一一查看。） 2 连接数多时，服务器端性能如何？ 3 对墙的动作有什么防范吗？假如被针对的话会如何？ 4 后续维护的动力如何？ 5 高级功能方面有考虑过吗？ 6 是自用或小范围使用，还是想做大推广？预想的用户群体是什么，开发者，还是一般用户？ 7 如果别人要入坑的话，理由是什么，什么地方有吸引力？

[enfein]

qeatzy 谢谢你对本项目的关注！

1 延时低吗，尤其是打开多个浏览器窗口时。（比如用Vimium，谷歌搜索后一次打开10来个后台窗口，然后一一查看。）

抱歉，我们目前做测试主要关注能不能翻墙，没有单独做过延时的测试。

2 连接数多时，服务器端性能如何？

当带宽足够时，连接数对服务器端性能影响不大。服务端的 CPU 和内存使用量都很小，带宽是决定因素。

3 对墙的动作有什么防范吗？假如被针对的话会如何？

对墙的防范主要是防止基于重放攻击的主动探测。我们的流量是完整加密的，而且添加随机填充字符，很难通过特征识别的方式发现。我们在设计协议时，参考了历史上 GFW 进行主动探测的案例，制定了针对性的防御措施。我们认为，TCP 协议有很小的概率遭到针对性攻击，UDP 协议被攻击几乎不可能。

4 后续维护的动力如何？

被抓之前都会一直维护的，即便我离开天朝。最近在自学移动开发，后续可能会做安卓客户端。

5 高级功能方面有考虑过吗？

请列举。

6 是自用或小范围使用，还是想做大推广？预想的用户群体是什么，开发者，还是一般用户？

希望社区成员因为喜欢这个项目而自愿推广。不打算靠这个挣钱，所以不需要商业推广。做这个主要是为有一定技术实力的人提供一个持续可用的翻墙工具。

7 如果别人要入坑的话，理由是什么，什么地方有吸引力？

这句话需要时间来检验：在天朝完全变成局域网之前，我们打算做一个一直可以稳定使用的翻墙工具，不需要担心六四或者其他敏感时期梯子失灵。

[qeatzy]

> 5 高级功能方面有考虑过吗？

请列举。

5.1 one port multiple user 5.2 multiplexing，使用多条连接突破ISP/GFW对单连接的限速。 5.3 ping不通，ssh也连不上的情况下，但某些端口仍然可建立连接，aka，间歇性tcp阻断，如何支持翻墙。 以及可能已经实现了的 5.4 udp支持，udp tunnel 5.5 对抗dns污染的手段 5.6 路由规则，pac.txt等 5.7 客户端支持同时连接多个服务器端，选择线路质量好的。 其他

[enfein]

5.1 one port multiple user 5.2 multiplexing，使用多条连接突破ISP/GFW对单连接的限速。

这两个已支持。

5.3 ping不通，ssh也连不上的情况下，但某些端口仍然可建立连接，aka，间歇性tcp阻断，如何支持翻墙。

可以尝试 UDP 协议。

5.4 udp支持，udp tunnel

已支持。

5.5 对抗dns污染的手段

我们的期望是，socks5 传给我们域名，DNS 解析交给服务器端进行。服务器端在墙外，GFW 无法进行污染。

5.6 路由规则，pac.txt等

这个打算交给浏览器来实现。我们这里就不引入复杂的逻辑了。

5.7 客户端支持同时连接多个服务器端，选择线路质量好的。

支持同时连接多个服务器端。自动选择线路这件事，写出一个好的实现可能会比较难。暂时在酝酿阶段，没有具体的时间表。

[enfein]

5.2 multiplexing，使用多条连接突破ISP/GFW对单连接的限速。

更正一下，我先前的理解有误，这个是不支持的。如果浏览器只发起了一个连接，代理也只用一个连接。大多数站点和浏览器针对连接的使用会做优化，代理不了解用户的意图，盲目做 multiplexing 不一定能取得好的效果。

[huyi51462]

最近封的很厉害,大佬你这个有被封吗

[qeatzy]

自己写的协议，稳的很，3楼里5.3的情况下都稳的很。思路见 shadowsocks/shadowsocks-org/issues/196#issuecomment-1130808838 shadowsocks/shadowsocks-org/issues/183#issuecomment-1130801232 长期来看，加域名可不是什么好点子，SNI阻断安排的明明白白的，cf之类迟早被玩坏，不信，看泉州就是。

[huyi51462]

仔细看了一下你这个的设计思路,如果全密文感觉是很稳啊,没有任何特征 研究下试试

[qeatzy]

@huyi51462 我不是本项目作者。（好像造成误解了，[逃])。

我是自己写的协议，长连接实现的，仅tcp，有握手，非对称加密握手，ecdh，类似ssh。思路见上述链接。我的本意是，这事本来不应该这么左支右绌的。你仔细看下5.3是什么。

不同于本项目，我的优先级是 1 低延时 （绝对的优化重点） 2 极端环境下的高稳定性 （5.3） 3 其他

【多余的话，权当胡言】 我的看法是ss和vmess主要不是因为DPI，而是因为头部，如果你是运营商要决定QoS，你会用所有流量的画像来做决定，还是第一个（或前几个）network layer packet来做决定。It's a PITA that so obsessed about replay attack and deprecating non-AEAD ciphers while missing the whole point. It's not that defense against replay attack not important, but how some guys treat it is plain wrong, since probing is the second/third step of your opponent. 善行无辙迹，共勉。

[qeatzy]

...... 盲目做 multiplexing 不一定能取得好的效果。

@enfein 最近实现了multiplexing，效果挺好，当然链路状况差的时候可能会增加延时和失败率。 实现参数,3 tunnel multiplexing.

// https://www.speedtest.net/result/13266865xxx PING 191 ms DOWNLOAD 41.95Mbps UPLOAD 2.67Mbps tested with vps year payment 10+$. before multiplexing DOWNLOAD 10+ to 40+, depending on time. local network bandwidth 100M. or 200M? at least no worse. :‑) still working on it.

I agree with you that it's probably not worth to even think about multiplexing unless long running tunnel is used, but in that case it will incur a lot more challenges and require protocol overhauls rather than monkey-patching.

== Edit another test PING 197 ms DOWNLOAD 68.67Mbps UPLOAD 2.73Mbps

== Edit Edit tests with proxy switch off PING 2ms DOWNLOAD 65.24Mbps UPLOAD 31.55Mbps PING 3ms DOWNLOAD 51.70Mbps UPLOAD 35.29Mbps PING 3ms DOWNLOAD 46.60Mbps UPLOAD 35.81Mbps PING 4ms DOWNLOAD 64.79Mbps UPLOAD 42.78Mbps

[enfein]

qeatzy 你好，

能看出来你对各类翻墙协议有深入的研究。我这个 mieru 项目倒是没有针对低延迟和网络性能做过多的优化。最早设计协议的时候，更多考虑到的是安全可靠，如果加 multiplexing 要推出下一版协议才可以。如果你有成型的项目，可以开源出来让大家一同受益。

[enfein]

huyi51462 你好，

我六四期间用 mieru 没有感觉到任何差别。mieru 的 TCP 和 UDP 协议都是不传输任何明文数据的，包括报文的长度也被加密，另外所有的加密都有 AEAD 防篡改验证，就从数据包本身来说 GFW 是没有什么办法的。GFW 可以选择使用白名单，把所有看不懂的协议都封掉，不过加速到那一步还需要一些日子。

[qeatzy]

成型的项目倒没有，尚在prototype阶段，也没有开源的打算。能在开源项目的项目中贡献微薄之力就很满足了（比如本项目enfein/mieru，e1732a364fed/v2ray_simple, ShadowsocksR-Live/shadowsocksr-native, txthinking/brook, geph-official/geph4，p4gefau1t/trojan-go, v2ray, ss/ssr。

比起大大+小白的模式。我更乐见百花齐放，授人以鱼，不如授人以渔。我眼中最好的社区是思维的碰撞，各种实现层出不穷，并且能相互支持，而不是圈地自萌+相互攻诘。相关的开源项目，我的看法是，共同的道+有能力有意愿的一群人+大众的收益和知识眼界的提升，这种模式，远好于几个独裁者+路人+粉丝+黑子的模式。

至于效果（稳定性，cpu & mem utilization，network delay & throughput）这个使用起来差异很大，很多时候往往是如人饮水，冷暖自知（很多时候人们无法共情别人的遭遇和困难，尤其是这里）。而我能做到的只不过是，知道自己要什么（绝对的低延时+no fuss+no headache），而不满于ss/v2ray，自己造了个趁手的轮子而已。

至于加速主义，站队等相关的话题，我是不参与讨论的，我觉得，一个人知道自己要什么，愿意放弃什么。

你如果在技术/协议方面感兴趣的话，后续可以互相实现验证。

Edit 补充下我为什么自己写，（顺序与重要性无关，想到哪写哪） 1 我不喜欢relay出去来支持http，说的就是shadowsocks-windows下的privoxy，或linux 的tproxy等。 所以自己实现支持http/https/socks5，就几十行代码的事。（我想要低延时，蚊子肉也是肉） 2 我非常不喜欢非要引入域名。（此条请忽略，后续不会讨论这点，因为这点上不同人观点差异非常大。）你这样搞收益不大，却损失了别人的利益，想想那些好端端建站的，结果被连带误封，这点不是其中的一个原因吗。至于收益，引入域名就不会被q，怎么可能。谷歌怎么被封的。什么，你说域名不是重点，https才是重点。那你告诉我，tls handshake之后就是paylod tls records, 5 byte header + usually aes-gcm encrypted data, how can you easily differentiate from other traffic (highly also aes-gcm or similar encrypted data)? why do you not adopt tls without domain name or other weaker version (certificate) to achieve you promoted stability gain? Make you friend happy, you foe cost higher, instead of hurt your friends and make you opponents' action very simple. 最常用的域名你的对手能区分，最常用的ip你的对手能区分，哪个能引入更大的成本，不考虑下？ 3 稍微掉个包或被加个延时，就一直转啊转。 4 出了问题你都不知道什么原因，去反馈被“积极分子”嘲讽+质疑，如此卑微。 5 using persistent tunnel approach to reduce 1 rtt 6 use ecdh handshake to provide support for forward secrecy, rather than being told you don't need that, or monkey patching. 7 support multiple-user multiple port natively at both client & server side, do whatever combination you want to achieve your desired effect. 8 support traffic dispatch at many layer, someone not need it doesn't mean others don't. 9 being simple, no simpler, do not being overly complex (one reason I do not keen on v2ray). 10+ 其他

[qeatzy]

@enfein 后续发现，上述性能的提升跟multiplexing关系不大，目前看来主要是程序优化的结果。

针对单个client connection对接多个tunnel multiplexing意义不大，反而代价明显。只对运营商只针对ip/port[TCP/IP的四元组限速才有意义，这种场景应该是越来越少。而代价很明显，需要synchroniztion，只要有一个tunnel慢/丢包就会极大的影响。食之无味，弃之可惜。

真正有用的multiplexing是针对多个连接的multiplexing，类似HTTP 2的那种长连接，但这就要引入packet的概念，类比tls record，像ss一样无脑socks stream肯定是不行的。

Edit This is how I react to senerio 5.3, rather that switch to udp, hoping udp lessly used or less easily block meanwhile sacrifice performance due to low qos，this is my view, even if 99 out 100 connection establish failed, the successful tunnel could be used, cannot ssh, then ssh through your tunnel to reach vps. guess how many time/energy need to try 100 connections? fear for ip ban? you are already banned. Or all tcp whose ip originate outside will be drop, will this happen? in this last resort, do you think ip-based or domain-base more likely to happen?

[enfein]

@qeatzy

我想仔细讨论一下网络性能和反封锁两个话题。

网络性能

如果用 TCP 协议，我的意见是，一个用户连接在代理中对应多个连接意义不大。以我有限的实际体验，还没有察觉到运营商对 TCP 单个连接有限速。

反过来，如果把多个连接在代理里面对应一个连接，能带来什么好处呢？减少握手的一次 RTT，另外跳过 slow start 过程，这些对降低延迟提升性能是有价值的。但是这种做法真的没有问题么？TCP 里面的数据是有序传输的，如果遇到了比较慢的请求，或者单纯的网络丢包，后面的应答就算准备好了也要等待，这是会降低性能的。想一想浏览器发起 HTTP 的时候为什么要用多个 TCP 连接。你想在 TCP 里面实现 UDP？没用啊，只要你的应用必须消费有序的的字节流，卡顿是无法在 TCP 协议内部解决的。我不认为 shadowsocks 那样 TCP 连接 1:1 对应有什么根本的性能问题。这种 1:1 对应倒是会暴露一些流量特征，mieru 尝试用 padding 来减少这种特征，但是做不到根本性消除。

反封锁

我不喜欢用域名，TLS 不是为了反封锁设计的。

我和你的想法不同，我认为面向连接的 TCP 协议是有利于封锁的，加速到后期还是要依靠 UDP 协议。事实上 mieru 最早推出的就是 UDP 协议，无奈运营商的 UDP 限速对用户体验伤害太大，所以后来出了 TCP。但是 TCP 能用几年，这件事不好说。

[qeatzy]

一个用户连接在代理中对应多个连接意义不大。

这正是我上条回复说的。

如果把多个连接在代理里面对应一个连接。

My current approach. If described more accurately, it's more than thant, N local <-> M tunnel <-> N remote, as said at shadowsocks/shadowsocks-org/issues/183#issuecomment-1130801232. Even under extreme conditions like 5.3 above, I can ssh to vps through my own tunnel, demonstrating high availability per se.

5.3 ping不通，ssh也连不上的情况下，但某些端口仍然可建立连接，aka，间歇性tcp阻断，...

senerio 5.3, already stated above.

我不喜欢用域名

me too. 🍺

udp

Yes, as non-connection based protocol, it is indeed harder to targeted at, but it's already targeted by other ways (eg, qos). Yeah, if tcp way is not even possible, I will choose to use udp too, just like you choice. But who will choose worse approach if better way do exist. Though udp adoption do grow with quic deployment etc.

本来不应该这么左支右绌的。

This is my main stance. I just proved tcp works exceptionally well, unlike many stated. Similarly, as for ip/domain, I goes for ip, not domain, contrary to some's belief I think ip approach will outlast domain ones, and incur larger cost, which is another gain. I just proved it's indeed possible to achieve great availability & performance with tcp/ip, even under extreme senerios. That's my main point. I wanna tell one fact, it's not mission impossible, it's just harder to implement. It works well even when ping unreachable, nc 80/433/etc all failed. It achieves 50Mbps, when local is 60-70 at PC. And acheive 220+Mbps at android with 60-70 local, yeah, a suprise faster than local, I was told I do have 1Gbps link, but doubted it untill yesterday.

slower links incur problem.

Think about this, which probability is higher, a) tens of tunnel, already proved to work, periodically check state and re-instantiate b) tens or even hundreds of new connections. Which is more likely impacted by slow link or timeout or drop packet. Which one is more likely be affected by inspection if inspection do happen. Which one is more easily fingerprinted. Whose attack surface is larger. If you are force/required to drop packet, which one will you choose, active ones or new ones. Not to mention other security/performance gains. Yeah, they all said this is too hard, that is not needed, those are no good, but let's admit it, they are not the authors who design & implement most of it. Some may be just pretending as big lao and enjoy the feeling, some are not affected since his/her link is good or drinking coffee in San Francisco, some do want to help but cannot afford the time/proficiency or daunted by sheer complexity, v2ray, cough,cough. Even under 5.3 senerio it must not sacrifice any aspect of delay/speed, and any weaker version is unacceptable by my view, this is how I work/test it. Treat seriously about the risk/gain, the v2 author Victoria Raymond disappear years and seems no one cares. Whose treatment IMO is still worser than breakwa11. The blogger mentioned in you docs about securely surfing is gone too.

Implentation detail You can wrapper around connection read/write, set time before & after syscall, maintain a persistent thread to check periodically, if now - t_succcess is too large, it suggest something might go wrong, and some action could be taken. Sidenote, as micro-optimization, do not naively get time by each op, only get time in the persistent thread since fine grained time not needed, unnecessary syscalls do affect performance. You said right, it's a lot more muddy approach and playing bad with those unprepared. It's a tradeoff of pain & gain. There is no such thing as accelerationism, it's always cost/benefit.

If handshake do adopted, remember it's a different security model from tls, no certificate needed. it's more like 2 gpg holder verify each other. And please do not make mistake as v2ray to mix identification & authentication as one, which greatly hinder transparent support multiple client with same authentication, which might works with socks like approach 1:1 mapping, but a big no-no with multiplexing.

Seems like you have some minor mis-understanding of multiplexing, you don't need to simultaneously tuck multiple local active connections into one tunnel, you can re-assign connection to a tunnel only if it's being not occupied. Though, in practice, too many/few tunnels is both not good. And if connection number goes high, each tunnel will be used by multiple connections. I just stick to round-robin before better way exists. In summary it's like http2, but M tunnel instead of one.

[qeatzy]

Implementation details There are several challenges to successfully implement a mutiplexing proxy with high availablility and mximimal performance. Just name a few,

• early detection of unhealthy link and graceful teardown
• memory efficiency
• proper synchronization

proper synchronization

with multiplexing, each tunnel reader need to get connection id for each packet. what if 100+ tunnel's simultaneously getting id interleaved with serveral newConn operations, with the total upload link speed reachs 100Mbps. There are ways to exclude connection id from packet, but then each tunnel become stateful and the burden shift to synchronization client & server side, and also reduce the utilization rate of each tunnel.

memory efficiency

to reach for maximal speed, you need to separate read from write, which incurs transfer ownership of memory, and default gc/malloc way is far from efficient when network speed utilization is high, you need to implement arena/allocator somewhere.

early detection of unhealthy link and graceful teardown

similar to what you already said.

[qeatzy]

As for high availability. There seems two ways, the trojan way and the ss way. But they are the same at a deeper level.

Though implemented differently, they actually share same premise, -- hoping being recognized as something and do not block. ss & plain vmess hopes to being recognized as random/unknown protocol. trojan hopes to being recognized as normal web traffic.

Sadly both is intrinsically flawed. ss & vmess's fixed value at fixed postion in cleartext do trigger accurate detection. The maintainers choose to react to equal to any request instead of fix the flawed protocol design.

The trojan family (eg, v2ray+tls etc) pretend to be normal traffic, but their users do not hold prominent domains and highly prioritized certs. With the adoption of ESNI seems gloomy. The pretending is just illusion.

I pointed out elephant in the room and introduce the unexplored approach of multiplexing at here. But the reaction is all but negative.

Actually it's inaccurate to say multiplexing being unexplored, since the good old ssh tunnel is precisely multiplexing, though the result is far from ideal due to single point of failure. However with M tunnel, you can get all the benefit of ssh and http2. But sadly, no one has agreed with me yet.

As stated early, even if your 99 of 100 handshake failed due to drop packet or timeout, the successful one will serve you for at least minutes. Even in the most extreme accelerationists' view, it's hard to envision all tcp connection to a blacklisted ip set will never succeed.

That's what I recalled the second premise. A truely yang mou.

[qeatzy]

Similar to my choice of tunnel binding, the chrome guys adopt "late binding" since early binding is apparently sub-optimal, the final SPDY solution interleave mutliple http stream with one active socket, which is the same as my current approach, proabably with a more fine-grained session manager. Connection Management in Chromium source from gfwrev

[enfein]

@qeatzy

感谢你的长篇回复。看过之后，我意识到这个 M:N:M 的 TCP mesh 的确有可取之处。

关于之前你在 shadowsocks 社区的遭遇，我是完全可以理解的。shadowsocks 作为一个千万用户级别的产品，关于兼容性方面一定要小心谨慎。仅仅是为了应对主动探测把 infinite read 改成 one time read 这件小事，都会导致一大批 shadowsocks 实现无法与新版本一起工作。像你这种翻天覆地的架构变化，在社区看来一定是来砸场子的。

由于你这种想法与已有的各类翻墙软件的实现均大为不同，已有的软件要么出于兼容性的考虑不能改协议，要么会因为加入新协议变得更加臃肿（参考 v2ray），要么有兴趣但是出于理解水平和技术实力做不到期望的效果（比如本项目，另外，我真的很缺时间）。所以，最好的办法依旧是另起炉灶创立新的开源项目，用实际的用户体验赢得声望。事实上我作为近几年长期使用 shadowsocks / v2ray / trojan 的用户，清楚知道他们各自有哪里做得不好。做出一个我自己理想中的翻墙软件，是我维持这个项目的根本动力。

做开源翻墙软件最大的阻碍自然是安全。你之前也提到过几位先烈的结局，我这里不再重复。如果安全是一个核心的考量，那么我完全同意你开发出来自用，毕竟没有任何人有义务把自己通晓的牛逼翻墙姿势传播给其他人。但是在开源社区里面，talk is cheap，只有真正拿得出项目和代码的人才有发言权。

言归正传，我已经许诺了 mieru 这个项目会继续开发下去，无论我个人今后是否用得到。我在谨慎地衡量 M:N:M TCP mesh 的可行性，或许在将来，它会成为 mieru 支持的第三种协议。如果有那么一天，还望仁兄赐教。

[qeatzy]

One core reason yet unmentioned for the choice of multiplexing approach is, new protocol without authentication is not reliable and not recommended at all, as already suggested by third party anticensorship organization like xxxreport years ago. Non-handshake authentication is flawed, whereas without multiplexing asymmetric handshake is too heavy.

I can provide some core module implementation that are performance related as reference, if there will be someone that's determined to work on it and prepared for all kind of obstacles. Hope you make a good fortune and still has the determination. Let's hope for good and not expect the community being stagnant or go down.

As for the authority in this area of open source project and community, contrary to what you said, it's rather more complex, it's not more code more power of disclosure. Let's raise two kind of samples.

• rprx & breakwa11. Despite the code quality might not be top class and breakwa11's major was not IT, both do has many great innovations and many advocates, but still are forced to leave, the hatred is not just bottom up or due to plain characteristics or some sub-optimal behaviors, it's more complex, many parties in play, same/sibling project collaborators, other open source community like debian/fedora, cryptographic practitioners, network security guys, official net army disguised as outsiders or maybe even contributors, ji chang zhu, outsiders that support, outsiders chi gua, outsiders that oppose, outside never give it a shit. I don't speak for them or say they are perfect or without defects. What I want to point out is, there are far too few allies and way too more obstructions, some from insiders. Remember it a very complex system with intrinsic chaos behavior, sometimes even an unintentional action triggers many thing unanticipated.
• If you view the issue history and insight -> contributors -> commits, you will notice there are only 1 or 2 prominent contributors excluding the original developer, usually 1, at most 2 or 3, and many of these do not say much in the community. It's not who has the more proficiency or contribute more code has more power of disclosure. Like many things, it's will + skills + some prerequisite. The contribution is just prerequisite and not even have to be in the form of code.
• Another thing. Many if not all of the projects are from one single core author, ss/ssr/v2ray/xtls/trojan-go/naiveproxy. That's a sad truth every new open source projection author will have to face eventually, if it will ever get more adoptions.
• I think what I want to expose has already stated. Let's put it down. You can close it at you will.

[enfein]

还有一点想请教，为什么客户端和服务器之间一定需要 DH key exchange。

这里面涉及两个问题。

第一，为什么没有 DH KEX 的情况下通信的安全性或者隐匿性得不到保证？能否给出详细的论证或者参考资料？

第二，加入 DH KEX 会让翻墙协议更容易被识别。你之前提及，shadowsocks 的原理是模仿一种未知的协议，trojan 的原理是模仿 TLS 协议。这种模仿和混淆能成功，是因为未知协议的集合太大，GFW 无法进行有效的判别，另外 GFW 不容易分辨浏览器产生的 TLS 流量和 trojan 产生的 TLS 流量。如果用了 DH KEX 情况就不一样了。我假设 DH KEX 过程数据包是明文传输，那么你具体的实现一定会有具体的数据包的特征，这种特征和正常的 TLS 流量是容易分辨开的，导致你的翻墙协议被识别封锁。如果 DH KEX 是密文传输，相当于退回到了 shadowsocks 的模式，那么 KEX 的意义是什么？

[qeatzy]

My design, e: ephemeral, l: long term, s: server, c: client

• server packet 1: [padding][ID][e_pubkey_s][extra-random-payload-for-entropy][padding]
• client packet 1: [padding][ID][e_pubkey_c][extra-random-payload-for-entropy][padding]
• server has l_privkey_s, has a list of l_pubkey_c. similarly for client.
• both generate sign with [ID_s][e_pubkey_s][extra-random-payload][ID][e_pubkey_c][extra-random-payload][shared secretkey from f(e_pubkey_other,e_pubkey_self,e_privkey_self] and l_privkey_self
• s/c packet 2 [the signature], upon reception, do verification.
• ID is used for identification, l_pubkey for authentication.
• You can even interleave all info with other random or fake tls or as http header or whatever, as long relevant info provided.
• It don't have to be the first packet.
• It's not like ss nor tls, more akin to ssh/gpg.
• No certs chain, no cipher suites selection. Just agree upon.

My implementation detail

• do make ID seems random, while effectively avoiding collision.
• no fixed value & fixed position, if some aspect unavoidable due to some implementation details, plugin obfuscation when needed, being prepared for the worst even before start.

The key point is

• no fixed value at fixed point. If fixed length become problem, then fix it.
• support easy customization, every server/client can and should hold unique fingerprint.
• As for padding or extra fake characteristics, they are all optional but strongly advised against to using the same choice.
• forward secrecy.
• support multiple user with different authentication naturally, support multiple client with same authentication without mixing together.
• Since inner payload is usually aes-gcm encrypted similar to tls payload record. The only leaking characteristics is the handshake. With above setting, I don't see any substantial weak point, as long as the implementation is robust.
• Easy support for client to use multiple server at same time, just plugin a custom dialer/session-manager at client if do insist advanced usage.

[qeatzy]

I do not known much detail about trojan or trojan-go, but for v2's tls, it's easily detectable, just see the many relevant issues. What's worse, it's even different from other go program fingerprint, before a fix being applied, due to fixed setting of cipher suite.

That's why naiveproxy ever being created, to re-use chrome network stack to avoid that.

And there seems no strong interest in improve on this fingerprint part. Their premise seems it's too damaging to ban all go programs, thus current setting is good enough. Unless some standard library provide the needed functionality out of box, seems unlikely to improve upon or provide custom choices.

It's tls handshake being detected for v2's tls, since tls handshake has plain cleartext content which in predictable position, thus easily detectable. Sidenotes, that's where tor failed. -- If you do found design in previous post has some point said, please do make backup, I might delete it after some time, to avoid unnecessary debates. sha256sum doc/design-orig.txt 7565728d36ca7e9d002e612601c0af7e49313535133fdfb39b2229b31e442a09 doc/design-orig.txt wc design-orig.txt 24 292 2030 doc/design-orig.txt

As for the faking as other protocol, ss did succeeded, but that's history years ago and broken afterwards, similar story with v2's tls. newer version may improve on replay part, but suspect the motivation contain anything with detectability (fixed value at fixed position with predictable length). Also tls in tls do leak characteristics, due to predictable length of tls handshake.

[huyi51462]

@qeatzy 是的，搞错了，不过你的发言和回复对我来说也很有价值。 我也是被GFW搞烦了，所以想自己实现一个私有协议，TCP/IP这块基础比较薄弱，看看你们的聊天会给我很多启发

[huyi51462]

有的加密都有 AEAD 防篡改验证，就从数据包

@enfein 感谢解答我的疑问，就是比较担心GFW会不会直接封锁看不懂的大流量密文（google youtube），大概每月100G，如果会的话，自己写也没多大意义。

[huyi51462]

My design, e: ephemeral, l: long term, s: server, c: client

• server packet 1: [padding][ID][e_pubkey_s][extra-random-payload-for-entropy][padding]
• client packet 1: [padding][ID][e_pubkey_c][extra-random-payload-for-entropy][padding]
• server has l_privkey_s, has a list of l_pubkey_c. similarly for client.
• both generate sign with [ID_s][e_pubkey_s][extra-random-payload][ID][e_pubkey_c][extra-random-payload][shared secretkey from f(e_pubkey_other,e_pubkey_self,e_privkey_self] and l_privkey_self
• s/c packet 2 [the signature], upon reception, do verification.
• ID is used for identification, l_pubkey for authentication.
• You can even interleave all info with other random or fake tls or as http header or whatever, as long relevant info provided.
• It don't have to be the first packet.
• It's not like ss nor tls, more akin to ssh/gpg.
• No certs chain, no cipher suites selection. Just agree upon.

My implementation detail

• do make ID seems random, while effectively avoiding collision.
• no fixed value & fixed position, if some aspect unavoidable due to some implementation details, plugin obfuscation when needed, being prepared for the worst even before start.

The key point is

• no fixed value at fixed point. If fixed length become problem, then fix it.
• support easy customization, every server/client can and should hold unique fingerprint.
• As for padding or extra fake characteristics, they are all optional but strongly advised against to using the same choice.
• forward secrecy.
• support multiple user with different authentication naturally, support multiple client with same authentication without mixing together.
• Since inner payload is usually aes-gcm encrypted similar to tls payload record. The only leaking characteristics is the handshake. With above setting, I don't see any substantial weak point, as long as the implementation is robust.
• Easy support for client to use multiple server at same time, just plugin a custom dialer/session-manager at client if do insist advanced usage.

精彩，受教

[enfein]

@huyi51462 对于全加密流量，如果是 TCP 协议，理论上讲 GFW 可以做深度包检测并且掐掉看不懂的连接。不过这种白名单方法很容易误杀，一时应该不会启用。如果是 UDP 协议，没头没尾，GFW 除了丢弃所有 UDP 包没有什么很好的办法。

[huyi51462]

@enfein 大概思路明确了，这两天我的使用可能验证了你说的GWF的行为： 我PC上无论是linux上的tproxy，或者是win下v2客户端，TCP是直接给我断了的，已经建立的连接，也会被阻断， 我安卓设备上的v2rayNg,在封锁情况下发起的tcp连接也是被阻断的,但是已经建立的连接可以稳定使用 我想，这说明GWF会对V2协议握手做检测，也就是 qeatzy 大佬讲的对头部做匹配，这样应该可以搞掉9成以上的留学者了,也会对已经建立连接的流量做检测，但可能pc端和安卓端的一些操作造成了流量数据的头部特征差异

[qeatzy]

最近想尝试udp的起因是换了机房后，速度骤降，从原来的40-200Mbps降到0.5-2Mbps（极少数空闲时能达到40-50Mbps，但上升非常缓慢，之前上升非常快，很快就达到峰值），用WinMTR查看路由，丢包率达到8-13%。

初步实现的过程中，用udp传输一个64KB的文件，上行丢包率小于1%，最大不超过2%，下行丢包率始终在10以上，42K-57KB，以49-52KB最为常见。实现为echo server。

本地 ./client <input >output

服务器端 ./server >output

从响应速度来看，去掉 >output 之后本地结果瞬间出来（远小于1秒，看不到打印过程），而服务器端terminal的输出在几（4-5）秒之后才逐渐完成。

为排除其他原因，切换为一个hk服务器测试，本地1-2秒打印完成，服务器端也几乎同时完成。

参数 服务器1 美西 1Gbps 服务器2 hk 3Mbps

从上述分析，我认为在这种情况下udp大有作为，可大幅度提高性能。 我认为上述性能下降的原因在于高丢包率+TCP的传输的AIMD性质导致速度始终非常低。 我个人认为KCP在这里帮助有限，而QUIC能有一定的提升（multiplexing stream, 减少了队头阻塞）。 使用QUIC的现存实现有 HyNetwork/hysteria 我目前的看法是QUIC由于

• 复用减少了队头阻塞，
• 至于重传机制能比tcp bbr之类有什么提升，尚不太清楚。

而KCP实际上并不是为上述场景优化的，从它的设计思路看，以及从我之前的使用体验看（1-2年前搭配ss使用过，效果一般），好像从你的回复来看也是如此（你应该一直是KCP实现udp的吧，我记得你好像说udp不太好用）。

而且，随着网民数量的增多和外网需求增大，而国际带宽有限，至少从运营商的角度来看，丢包是几乎无代价但有奇效的手段。所以，上述场景应该是会越来越多的。

我的看法是，降速的源头是高丢包率+TCP本身的性质。那么只有通过udp来自己实现，针对性的应对。两种方案，一种直接使用成熟方案，一种自研。前一种，quic是可行的方案，目前hysteria据说效果不错HyNetwork/hysteria/issues/321，但我不认为这是普适或长久的，尤其是大规模的用户使用后。那时候就需要使用quic，但需要魔改或调参，或者自研。前者的好处在于起点高，实现难度低，缺点在于复杂度高，想调优事实上很难，也很难孤立某一部分做单独的尝试或测试。自研的缺点在于，起点低，实现难度很高，达到跟成熟方案开箱用的同等效果需要很大功夫，但优点在于灵活性高、方案复杂度可以降到相对很低，后续调优和改进能非常容易实现。我决定采取后者，放弃kcp，实现类似quic的multiplexing方案。

[gaukas]

对于全加密流量，如果是 TCP 协议，理论上讲 GFW 可以做深度包检测并且掐掉看不懂的连接。不过这种白名单方法很容易误杀，一时应该不会启用。

@enfein Please refer to this tweet from Great Firewall Report.

Edit: I am still waiting for their promised detailed report about this new behavior.

那时候就需要使用quic，但需要魔改或调参，或者自研。

I'd agree with @qeatzy and in fact, I may start a new project in @refraction-networking regarding this issue. It would be like https://tlsfingerprint.io/ and utls but for QUIC. No ETA for now though...

[qeatzy]

@enfein Please refer to this tweet from Great Firewall Report.

我正是切换到洛杉矶机房（不是vultr）后速度骤降。

I'd agree with @qeatzy and in fact, I may start a new project in https://github.com/refraction-networking regarding this issue. It would be like https://tlsfingerprint.io/ and utls but for QUIC. No ETA for now though...

Really looking forward to it.

maybe related https://github.com/trojan-gfw/trojan/issues/597 https://github.com/v2fly/v2ray-core/issues/1823 https://github.com/v2fly/v2ray-core/issues/1380

[gaukas]

maybe related https://github.com/trojan-gfw/trojan/issues/597 https://github.com/v2fly/v2ray-core/issues/1823 https://github.com/v2fly/v2ray-core/issues/1380

Thanks for the information. From my perspective,

• I did see user reports saying that vmess (which is not protected by TLS) of V2Ray is suffering from communication quality degradation or even TCP port blocking, starting early June this year. Which I believe is related to the behavior mentioned by @gfw_report.
• Reports regarding trojan and other TLS proxies attract my attention more. So far there is no evidence indicating that GFW is utilizing TLS-fingerprint-based attack.
  • Correct me if I'm wrong but the last well-known attack against TLS proxy in China was about a year ago? I kept a very crude record about it here: https://github.com/Gaukas/GFW-2021Summer-TLS-RST-Incident
• There might be some technic used by GFW to identify a TLS proxy server.
  • A fingerprint-able ClientHello used by the proxy client (fixed by only XTLS-core among popular implementations/libraries?)
  • A different behavior from the Server.
    • e.g., server response when a client sends plain-HTTP requests to HTTPS port. A 400 Bad Request is expected from a real HTTPS server, but the server from https://github.com/trojan-gfw/trojan has different behavior. Known to be fixed by setting https://github.com/p4gefau1t/trojan-go fallback_addr in configuration.

I would expect QUIC to take over TLS(-via-TCP) at some point. Then UDP is definitely the way to go...

Are we kinda getting off-topic here😄? Discussions are always welcomed tho, as long as everyone feels comfortable.

Back to the thread now, I will be excited to see more censorship-proof ideas. Mimicking a popular protocol (say HTTPS) is one of the ideas but what are the ways otherwise?

[qeatzy]

If you do poke around the issues, you will find many blocked that use ws/tls, some even under nginx. Among them, excluding misconfigured ones, there do exists many being blocked.

The authenticity of 接近3W个*ray 服务器 看看有没有你的 is unknown, but it should not all be accused of one-click script or misconfiguration or just large traffic.

Yeah, there is no clear evidence of tls fingerprint being used, but it's possible, as the POC in cited links from one of above issue. I'm not too positive to leaning on hoping not being exploited view, as the ss/vmess history has told. A protocol being popular doesn't make it immune to exploitation, this the not the perceived use case for tls. Since ESNI no wide adoption, the fingerprint part will eventually be used, unless the cost too high (efficiency & accuracy).

Yes, that's kinda of off-topic. But, in this community, IMHO, one of the most difficult thing and constantly obstruct ideas/impl is motivation/consensus.

Back to the thread now...

I did propose in previous discussions based on tcp & persistent tunnel. But since it's too long to read. Just list some recap https://github.com/enfein/mieru/issues/8#issuecomment-1155064457 https://github.com/enfein/mieru/issues/8#issuecomment-1154623819 https://github.com/enfein/mieru/issues/8#issuecomment-1159453773 https://github.com/enfein/mieru/issues/8#issuecomment-1159454805

As for the aforementioned tcp solution, which still works quite decently, but can not do anything with the new context of high packet loss due to inherent nature of tcp, that's the motivation why i started to look around udp solution.

Mimicking a popular protocol (say HTTPS) is one of the ideas but what are the ways otherwise?

imho any serious design should not rely on faking solely, though it should provide custom faking with higher priority, but the focus should be increasing the variability of fingerprints, not for detectability. (decrease the accuracy of inspection+detection thus increase the net cost, rather than doing it contrarily as the current trend of emerging on tls)

[enfein]

@qeatzy KCP 基本上是 TCP 改了参数并添加了部分 TCP 扩展。因为 KCP 返回给应用程序的数据是有序的，在丢包率高的情况下，一条连接的性能会大幅下降。我还没研究 QUIC 的多路复用是通过怎样的机制来回避这个问题。

[qeatzy]

@qeatzy KCP 基本上是 TCP 改了参数并添加了部分 TCP 扩展。因为 KCP 返回给应用程序的数据是有序的，在丢包率高的情况下，一条连接的性能会大幅下降。我还没研究 QUIC 的多路复用是通过怎样的机制来回避这个问题。

It's promising, but not yet verified, especially in wilder situations. Multiplexing do decrease the impact of HOL blocking, but inside one stream, it still be affected by high packet loss, though unsure of quic's retransmission compared to tcp+bbr in this context. That's why i say it may need refactoring/modification, designing a simpler/more-compact one might be simpler in the long run. Serialization is must, quic just move it from connection level to stream level and reduce the probability of HOL blocking, udp is like a lightweight ip, I use udp since it's way simpler than ip, and switch to ip has no real improvement.

[gaukas]

Since ESNI no wide adoption, the fingerprint part will eventually be used, unless the cost too high (efficiency & accuracy).

I agree with you on this one. The SNI extension of TLS defeats the purpose of mimicking as the cost of blocking a not well-known ServerName isn't high at all.

increasing the variability of fingerprints

decrease the accuracy of inspection+detection thus increase the net cost

But what if a strict censor may block all unrecognized traffics in real-time? 😄 Like what they did in June against Shadowsocks/Vmess. My research group is actually working on identifying the mechanism of how is Shadowsocks and some other protocols blocked in real-time. At the moment it might be still a pattern-based attack, but there's no telling if a censor has a compelling reason for not doing a general attack.

My 2¢: As long as random traffic worths literally nothing to censor parties, they would have a good reason to kill it. Convincing the censoring party that proxy traffics are actually innocent is really important for increasing the survivability of a proxy protocol. A protocol white-list could be cheap for a government-backed censoring party.

[qeatzy]

But what if a strict...

see below quote

As stated early, even if your 99 of 100 handshake failed due to drop packet or timeout, the successful one will serve you for at least minutes. Even in the most extreme accelerationists' view, it's hard to envision all tcp connection to a blacklisted ip set will never succeed.

That's what I recalled the second premise. A truely yang mou.

Another quote

There is no such thing as accelerationism, it's always cost/benefit.

Edit: Sidenotes, there are thousands neighbors for the affected vps in the aforementioned *ray-list. And none before switching region.

Like what they did in June against Shadowsocks/Vmess.

It's not just vmess/ss, some ws/tls affected if not too many. But with the wider adoption of tls, the performance of faking/being tls will just degrade further. Remember, both google & no-one-care blog site are blocked, why not your free site with bad name & far fewer visitors. Forget about protocol temporarily, and think like a non-technician, it will help figure it out more clearly.

As long as random traffic worths literally nothing to censor parties ...

The persisitent tcp tunnel solution works exceptionally well, even in last two months, even when all port blocked & cannot even ssh or http/https get. (just ssh throught the tunnel) And I don't think it's hard to plugin faking tls, or even being tls. The deeper problem is being tls is not panacea. Since most trojan family faking site using non-prominent domain name & free or low priority certs, aka it's not in domain whitelist, it's far too easy to block, actually simpler than ss/vmess. It's very easy & practical to maintain domain whitelist -- easier than blacklist, but quite hard to maintain ip blacklist/whitelist.

[gaukas]

think like a non-technician Since most trojan family faking site has non prominent domain name

I DON'T CARE WHAT'S THE COST, JUST KILL'EM ALL! 😆

Seriously. if so, all niche sites, IPs, and of course, protocols are blockable. We really need to deploy Conjure to all ISPs!

[qeatzy]

I DON'T CARE WHAT'S THE COST, JUST KILL'EM ALL! 😆

But they will decide based on cost/benefit. As before.

Seriously. if so, all niche sites, IPs, and of course, protocols are blockable. We really need to deploy Conjure to all ISPs!

I disagreee. As just edited after you reply, it's hard to maintain ip blacklist/whitelist. (there will always holes in the ip dimension & protocol dimension.) But with wider adoption of tls, it might change from domain blacklist to whitelist or some hybrid. The cloudfare cdn is impacted by this emerging on tls trend, and many regualr sites are already affected, those are just regular sites, not fakes, some even with large number of visitors.

[gaukas]

cost/benefit

What could be the cost of having a whitelist allowing only popular protocols to a well-known IP (owned by a mainstream CDN provider?) with a good ServerName?

[qeatzy]

What could be the cost of having a whitelist allowing only popular protocols to a well-known IP (owned by a mainstream CDN provider?) with a good ServerName?

The ever-changing nature & the bureaucracy.

[gaukas]

and many regualr sites are already affected

It's never easy to minimize the collateral damage I'd say. Unfortunately.

And IMO, from the very beginning, the reason for mimicking a popular protocol is always to introduce heavier collateral damage as a cost to censoring parties.

[qeatzy]

And IMO, from the very beginning, the reason for mimicking a popular protocol is always to introduce heavier collateral damage as a cost to censoring parties.

If works, yes. If not works, no. It's a knife meant to deter instead of stab.

It's akin to threaten someone, if it will not listen, it's futile or you are saying the wrong thing. 刑不可知，则威不可测。此事亦如是。 The non-characteristics route is not dead end yet. Though I'd like to see both survived, same for tcp & udp.

innocent is really important for increasing the survivability of a proxy protocol

always to introduce heavier collateral damage as a cost to censoring parties.

But https/domain name is by its nature very biased toward giants, thus the average cost of remove long tail is actually minimal. That's why whitelist is actually simpler than old blacklist for domain name.

[qeatzy]

Closing.

If you guys want to debates. Go to another issue. mieru/issues/14

Just some quote.

But, in this community, IMHO, one of the most difficult thing and constantly obstruct ideas/impl is motivation/consensus.

Some may be just pretending as big lao and enjoy the feeling, some are not affected since his/her link is good or drinking coffee in San Francisco, some do want to help but cannot afford the time/proficiency or daunted by sheer complexity, v2ray, cough,cough.

If you do found design in previous post has some point said, please do make backup, I might delete it after some time, to avoid unnecessary debates.

What I want to point out is, there are far too few allies and way too more obstructions, some from insiders. Remember it a very complex system with intrinsic chaos behavior, sometimes even an unintentional action triggers many thing unanticipated.

Treat seriously about the risk/gain, the v2 author Victoria Raymond disappear years and seems no one cares. Whose treatment IMO is still worser than breakwa11. The blogger mentioned in you docs about securely surfing is gone too.

Last word. Why I insist write in English at first, to increase the cost and prevent unnecessary debates. But futile. You big lao. 😆

Farewell.

[gaukas]

Come on... I feel sad if you closed the issue feeling offended. I meant no offense. Just that a strong guarantee against censorship is always my (and my research group's) first priority. Being in an idealistic academic setting and my focus could be different from yours. But I do support some of your ideas. They are inspiring.

Open source is like democracy. There's no wrong idea, only good ideas or better ideas. However, all people tend to stick to what they want to hear.

I'd really like to proceed with my sincere debates/discussions but if you no longer feel comfortable with it... so long.

big lao

What's a big lao btw?

[qeatzy]

No, It's not being offensed, actually already expected many reactions, negative & positive, and the form. @Gaukas Don't be nervous, you are actually from the positive side, and I do appreciate the work of utls and alike, so are your constructive words, though disagree some, but different ideas should exist and being encouraged.

It's being deeply disappointed. Make me recall Lu Xun, RPRX, breakwa11, clowwindy. Yeah, I know. Some guys are saying. 你也配？Yeah, you are right. Sincerely.

If you guys want to discuss, then discuss, and do the real work. Instead of good old 党同伐异/拉帮结派. My group, I'm security doctor. I will might .... WTF. You are hurting the community and doing nothing real, just because you originate from some prominent group/Degrees??? I could say more. But I don't want to use same 党同伐异 ways.

The history just repeat. shadowsocks-rss/issues/38 Real positive intender like [phpbestlang] at shadowsocks-rss/issues/38 are too too too rare.

so long.

so long.

big lao

guys of this kind [renyidong] at shadowsocks-rss/issues/38

[qeatzy]

I closed the issue, because what I want to say is already fully discussed. Though mostly of them will deemed ignored & just laughed at. And to avoid unnecessary & meaningless post that's are pure attitude/standpoint.

Now since the job done. Let's finish it & move on.

[gaukas]

My group, I'm security doctor. I will might .... WTF. You are hurting the community and doing nothing real

~Now I'm feeling uncomfortable being accused~ LOL

[qeatzy]

I'd really like to proceed with my sincere debates/discussions but if you no longer feel comfortable with it...

Me too. But

• with the advent of issue/14, many more will come if ever get more popular, like what RPRX encountered.
• this post is already far too too long, to the extent hard to fully grasp.
• many ideas being said, it's hard to concentrate.

If you do want to talk about the design of protocol & performance tuning, new project/new issue might be a better place. Since different project might take different approach. I'm looking forward to see more open source projects (though probably will never happen, but let's just hope for good, and being prepared for the goodness).

[qeatzy]

However, all people tend to stick to what they want to hear. That's the root cause makes me feel sad. We human are built in that way but we are more.

@Gaukas Btw which aspect do you want to talk about, I'm kind of reluctant to participate some topics that inherently opinionated, but interested in real implementation that are efficient & simple & flexible so that everyone can and should own his/her own one. Bureaucracy is no good, complexity incurs bureaucracy & totalitarianism. Emerging on one approach & one implementation is dangerous. Just name a few, udp vs tcp, non-characteristics vs faking/being popular protocol, persistent tunnel vs 1:1 mapping, UoT or pure udp, detect based on header/first packets or dpi on all, detect server or detect client or detect traffic or other ways. It's too easy to stick to one.