wkrp
commented
4 years ago EDIT: Triggering an ESNI block from the outside no longer works, since 2020-08-13 06:32. See below.
Because the GFW's ESNI detection is bidirectional, you can easily experiment with the blocking yourself, even if you are located outside of China.
Here is a short payload that triggers blocking. (ffce is the ESNI extension that the GFW is looking for.)
160303003b0100003703035b72616e646f6d72616e646f6d72616e646f6d7261
6e646f6d72616e646f6d5d0000000100000effce000a53754772000000000000- Choose any responsive TCP host in China. It doesn't have to be port 443. For example, www[]().tsinghua.edu.cn:80.
- Start TCP-pinging the port using, for example, hping or Nping. You will see responses to your pings.
$ sudo hping3 -S www.tsinghua.edu.cn -p 80 HPING www.tsinghua.edu.cn (eth0 166.111.4.100): S set, 40 headers + 0 data bytes len=44 ip=166.111.4.100 ttl=102 id=56332 sport=80 flags=SA seq=0 win=2105 rtt=279.8 ms len=44 ip=166.111.4.100 ttl=102 id=63224 sport=80 flags=SA seq=1 win=2105 rtt=283.7 ms len=44 ip=166.111.4.100 ttl=102 id=19078 sport=80 flags=SA seq=2 win=2105 rtt=287.5 ms len=44 ip=166.111.4.100 ttl=102 id=1977 sport=80 flags=SA seq=3 win=2105 rtt=279.3 ms len=44 ip=166.111.4.100 ttl=102 id=41003 sport=80 flags=SA seq=4 win=2105 rtt=283.1 ms$ nping -4 -c 0 --tcp-connect www.tsinghua.edu.cn -p 80
Starting Nping 0.7.70 ( https://nmap.org/nping ) SENT (0.0752s) Starting TCP Handshake > www.tsinghua.edu.cn:80 (166.111.4.100:80) RCVD (0.3682s) Handshake with www.tsinghua.edu.cn:80 (166.111.4.100:80) completed SENT (1.0778s) Starting TCP Handshake > www.tsinghua.edu.cn:80 (166.111.4.100:80) RCVD (1.3598s) Handshake with www.tsinghua.edu.cn:80 (166.111.4.100:80) completed SENT (2.0798s) Starting TCP Handshake > www.tsinghua.edu.cn:80 (166.111.4.100:80) RCVD (2.3584s) Handshake with www.tsinghua.edu.cn:80 (166.111.4.100:80) completed SENT (3.0818s) Starting TCP Handshake > www.tsinghua.edu.cn:80 (166.111.4.100:80) RCVD (3.3580s) Handshake with www.tsinghua.edu.cn:80 (166.111.4.100:80) completed SENT (4.0840s) Starting TCP Handshake > www.tsinghua.edu.cn:80 (166.111.4.100:80) RCVD (4.3603s) Handshake with www.tsinghua.edu.cn:80 (166.111.4.100:80) completed
3. Send the trigger payload.printf '\x16\x03\x03\x00\x3b\x01\x00\x00\x37\x03\x03[randomrandomrandomrandomrandom]\x00\x00\x00\x01\x00\x00\x0e\xff\xce\x00\nSuGr\x00\x00\x00\x00\x00\x00' | nc -4 -v www.tsinghua.edu.cn 80
4. Now you will stop receiving replies to your pings for 120 or 180 seconds.
<details>
<summary>Python 3 program to generate trigger payload</summary>
<pre>
#!/usr/bin/env python3
# Generates a small TLS ClientHello that trigger's the GFW's ESNI detector.
# Writes output to the file minimal.bin.
#
# You can send the ClientHello with, for example,
# nc -v www.tsinghua.edu.cn 443 < minimal.bin
import struct
from scapy.all import *
load_layer("tls")
from scapy.layers.tls.all import *
# https://tools.ietf.org/html/rfc8446#section-3.4
def var(ceiling, data):
if ceiling < 256:
fmt = ">B"
elif ceiling < 65536:
fmt = ">H"
else:
raise ValueError(ceiling)
return struct.pack(fmt, len(data)) + data
# https://datatracker.ietf.org/doc/html/draft-ietf-tls-esni-01#section-5
def encrypted_server_name(suite, group, key_exchange, record_digest, encrypted_sni):
return struct.pack(">HH", suite, group) \
+ var(65535, key_exchange) \
+ var(65535, record_digest) \
+ var(65535, encrypted_sni)
clienthello = TLS(
msg = TLSClientHello(
gmt_unix_time = 0x5b72616e, # "[ran"
random_bytes = b"domrandomrandomrandomrandom]",
ciphers = [],
ext = [
# The GFW detector requires a syntactically valid
# server_name_extension, but the actual values it contains may be
# nonsense. Here we use a CipherSuite of 0x5375 ("Su"), a NamedGroup
# of 0x4772 ("Gr"), and zero-length key_exchange, record_digest, and
# encrypted_sni.
TLS_Ext_Unknown(type=0xffce, val=encrypted_server_name(0x5375, 0x4772, b"", b"", b"")),
],
)
)
TLS(bytes(clienthello)).show()
print(bytes(clienthello))
FILENAME = "minimal.bin"
open(FILENAME, "wb").write(bytes(clienthello))
print("output written to {}".format(FILENAME))
</pre>
</details>
gfw-report
klzgrad
Authors: Kevin Bock, iyouport, Anonymous, Louis-Henri Merino, David Fifield, Amir Houmansadr, Dave Levin
Date: Friday, August 7, 2020
中文版: 揭示和规避中国对加密SNI（ESNI）的封锁
This report first appeared on censorship.ai. We also maintain an up-to-date copy of the report on iyouport, gfw.report, net4people and ntc.party.
On 2020-07-30, iyouport reported (archive) the apparent blocking of TLS connections with the encrypted SNI (ESNI) extension in China. iyouport says that the first occurrence of blocking was one day earlier, on 2020-07-29.
We confirm that the Great Firewall (GFW) of China has recently begun blocking ESNI—one of the foundational features of TLS 1.3 and HTTPS. We empirically demonstrate what triggers this censorship and how long residual censorship lasts. We also present several evasion strategies discovered by Geneva that can be run either client-side or server-side to evade blocking.
What is Encrypted Server Name Indication (ESNI)?
TLS is the foundation of secure communication on the web (HTTPS). It provides authenticated encryption so that users can know with whom they are communicating, and that their information cannot be read or tampered with by an intermediary. Although TLS hides the content of a user's communication, it does not always hide with whom the user is communicating; the TLS handshake optionally contains a Server Name Indication (SNI) field that allows the user's client to inform the server which website it wishes to communicate with. Nation-state censors have used the SNI field to block users from being able to communicate with certain destinations. China, for one, has long been censoring HTTPS in this manner.
TLS 1.3 introduced Encrypted SNI (ESNI) that, put simply, encrypts the SNI so that intermediaries cannot view it. (To learn more about ESNI and its benefits, see Cloudflare's article). ESNI has the potential to complicate nation-states' abilities to censor HTTPS content; rather than be able to block only connections to specific websites, ESNI would require censors to block all TLS connections to specific servers. We do confirm that this is now happening in China!
Our Main Findings
0xffceextension is necessary to trigger the blocking.How Do We Know These?
We have made a simple Python program that performs the following:
The program sends ClientHellos with ESNI both inside-out and outside-in, while capturing traffic on both sides for analysis. The servers to which we send ClientHellos complete the TCP handshake, but they do not send any data packets back to the client, nor do they are first to close the connection. All experiments were conducted between July 30th and August 6st.
Details About the Blocking
Blocking by dropping packets, not injecting RSTs
Comparing the traffic captured on both endpoints, we find the GFW blocks ESNI connections by dropping packets from clients to servers.
This has two differences from how the GFW censors other commonly-used protocols. First, the GFW censors (non-encrypted) SNI and HTTP by injecting forged TCP RSTs to both server and client; conversely, we have observed no injected packets from the GFW to censor ESNI traffic. Second, the GFW drops traffic from server to client to block Tor and Shadowsocks servers; however, it drops only client-to-server packets when censoring ESNI.
We further note the GFW does not distinguish the flags of TCP packets when dropping them. (This is different from some censorship systems in Iran which do not drop packets with RST or FIN flags.)
The blocking can be triggered bidirectionally
We find the blocking can be triggered bidirectionally. In other words, sending an ESNI handshake from outside the firewall to inside can get blocked in the same way as sending it inside-out.
Thanks to this bidirectional feature, one can test this ESNI-based censorship remotely from the outside of the GFW without having control of any Chinese server. The GFW's censorship on DNS, HTTP, SNI, FTP, SMTP, and Shadowsocks can also be measured outside-in.
The GFW censors ESNI, but not omit-SNI
We confirm a TLS ClientHello without ESNI/SNI extensions cannot trigger the blocking. In other words, the
0xffcepayload of theencrypted_server_nameextension is necessary to trigger the blocking.We tested this by replacing the
0xffcein a triggering ClientHello with0x7777. After the replacement, sending such a ClientHello could not trigger the blocking anymore.This confirmation is important because some censors have been observed blocking any ClientHello message without the SNI extension, which would result in the blocking of both ESNI and omitting-SNI.
New extension values are not blocked
As informed by an anonymous reviewer on the riseup pad, the currently deployed ESNI uses extension value
0xffce(see Section 8.1). However, the newer ECH uses extension value0xff02,0xff03and0xff04(Section 11.1). We confirm no censorship has been observed on these extension values yet.Specifically, we replace the
0xffcein a triggering ClientHello with the values of0xff02,0xff03, and0xff04respectively. And no blocking is observed after sending such modified ClientHellos.A complete TCP handshake is required before triggering the blocking
We find a complete TCP handshake is necessary in order to trigger the ESNI blocking.
We conducted two experiments from the outside to a server in China. In the first experiment, without sending any
SYNpacket, our client sent one naked ClientHello message with ESNI extension every 2 seconds. In the second experiment, our client sent aSYNpacket and a ClientHello message with ESNI extension; but the server would not respond with any packet (not even to complete the TCP three-way handshake).In total, we sent 10 ClientHello messages in each experiment. The result shows no blocking or residual censorship was ever triggered; all ClientHello messages reached the server. This means a TCP handshake is necessary before triggering ESNI-based censorship. It also indicates, similar to the SNI-based censorship by the GFW, the censorship machine for ESNI is stateful.
The blocking can happen on all ports
We find the ESNI blocking can happen not only on port 443, but on all ports from 1 to 65535.
Specifically, we sent two ESNI handshakes in a row to the port 1-65535 of a Chinese server from the outside. For each port, we first sent an ESNI handshake; then after the connection timeout (after 20 seconds), we tried to complete a TCP handshake with the server again. If we do not receive any
SYN+ACKfrom the server the second time, we consider the censorship occurred on that port. As a result, the ESNI blocking was observed on all ports from 1 to 65535.This feature allows us to test ESNI censorship efficiently, as we can conduct testings on multiple ports of the same IP address simultaneously.
Residual Censorship
We find that the GFW employs "residual censorship" of ESNI connections. This means that, for some amount of time after triggering censorship for a given connection, it will continue blocking any connections with the same 3-tuple of source IP, destination IP, and destination port.
The precise duration of residual censorship appears to vary by vantage point. We observed residual censorship for 120 seconds at two of our vantage points, and 180 seconds at another vantage point.
Sending additional ESNI handshakes during residual censorship time does not reset the timer of the censoring machine. This is similar to the previously observed residual censorship on SNI-based blocking of the GFW. (Conversely, each additional packet set while residual censorship in effect in Iran resets the timer.)
These findings are partially based on the following experiment. From the outside, we sent one ClientHello message per second to port 443 of a Chinese server. The 1st, 2nd, and 121st TCP handshakes were accepted. All other handshake attempts were unsuccessful because the
SYNs did not reach the server.This result shows, similar to previously discovered SNI-based residual censorship, the GFW also employs residual censorship for ESNI. In addition, the fact that second handshake could complete means that it takes at least 1 second for the GFW to react and enable the blocking rules.
How Can We Circumvent the Blocking?
Geneva (Genetic Evasion) is a genetic algorithm developed by those of us at the University of Maryland that automatically discovers new censorship evasion strategies. Geneva manipulates packet streams—injecting, altering, fragmenting, and dropping packets—in a manner that bypasses censorship without impacting the original underlying connection. Unlike most other anti-censorship systems, Geneva does not require deployment at both sides of the connection: it runs exclusively at one side (client or server).
Geneva trains its genetic algorithm against live censors, and to date has found dozens of censorship evasion strategies in various countries. Geneva's strategies are expressed in a domain-specific language. Details of the language, along with the entire Geneva codebase, are available at the Geneva GitHub repository.
To learn more about how Geneva (or the Geneva strategy engine) works under the hood, see our papers or about page.
To allow Geneva to train directly against the GFW's ESNI censorship, we wrote a custom plugin that performs the following steps:
"test"to test if it has been censored.With this, Geneva discovered multiple evasion strategies in just a few hours. We describe them in detail below.
The Geneva strategy engine is open source on our Github.
All of these strategies can be run with our open-source Geneva strategy engine (repository). Since they operate at the TCP layer, they can be applied to any application that needs to use ESNI: with Geneva running, even an unmodified web browser can become a simple censorship evasion tool.
Note that Geneva is not designed as a general purpose evasion tool, and does not provide any additional encryption, privacy, or protection. It is a research prototype and it is not optimized for speed. Use these strategies at your own risk.
Evasion strategies
We trained Geneva over the span of 48 hours, both client- and server-side. In total, we discovered 6 strategies to defeat the ESNI censorship: 4 that work from the server, and 6 that work from the client.
The following are TCP-layer strategies that can defeat the ESNI censorship when applied exclusively at the client-side.
Strategy 1: Triple
SYNThe first client strategy works by initiating the TCP 3-way handshake with three
SYNpackets, such that the sequence number of the thirdSYNis corrupted.In Geneva's syntax, this strategy looks like this:
[TCP:flags:S]-duplicate(duplicate,tamper{TCP:seq:corrupt})-| \/This strategy performs a desynchronization attack against the Great Firewall. The GFW synchronizes on the corrupt sequence number, so it misses the ESNI request.
This strategy can also be applied from the server-side:
[TCP:flags:SA]-tamper{TCP:flags:replace:S}(duplicate(duplicate,tamper{TCP:seq:corrupt}),)-| \/Although this strategy makes it so the server never sends a
SYN+ACKpacket, this does not break the three-way handshake. During the three-way handshake, instead of the server sending aSYN+ACKpacket as usual, the server instead sends threeSYNpackets (the third with a corrupt sequence number).The first
SYNpacket serves to initiate a TCP Simultaneous Open, an archaic feature of TCP supported by all major operating systems to handle the case in which two TCP stacks send aSYNpacket at the same time. When the client receives aSYNfrom the server, the client sends aSYN+ACKpacket, and server responds with anACKto complete the handshake. This effectively changes the traditional three-way handshake to a four-way handshake. TheSYNwith the corrupt sequence number causes the GFW to desynchronize (but is ignored by the client), successfully defeating censorship without harming the connection.Strategy 2: Four Byte Segmentation
The next strategy we discover can also be used from client or server. In this strategy, the client sends the ESNI request across two TCP segments, such that the first TCP segment is less than or equal to 4 bytes long.
From the client-side, in Geneva's syntax this strategy looks like this:
[TCP:flags:PA]-fragment{tcp:4:True}-| \/This is not the first time Geneva has discovered segmentation strategies, but it is surprising that this strategy works in China. The Great Firewall has been famous for its ability to reassemble TCP segments for almost a decade now (see brdgrd). The TLS header is 5 bytes long, so by segmenting specifically the TLS header across multiple packets, we hypothesize this breaks the GFW's ability to protocol fingerprint ESNI packet as TLS. This has interesting implications for how the GFW fingerprints connections: it suggests the component of the GFW that performs connection fingerprinting cannot reassemble TCP segments for all protocols. This theory is supported by other segmentation-based strategies identified by Geneva in the past (see this paper).
This strategy can also be triggered from the server-side. By reducing the TCP window size during the 3-way handshake, a server can force the client to segment their request. In Geneva's syntax, this can be accomplished with:
[TCP:flags:SA]-tamper{TCP:window:replace:4}-| \/.Strategy 3: TCB Teardown
The next strategy is a classic TCB (TCP Control Block) Teardown: the client injects a
RSTpacket with a broken checksum into the connection. This tricks the GFW into thinking the connection has been torn down.In Geneva's syntax, this strategy looks like:
[TCP:flags:A]-duplicate(,tamper{TCP:flags:replace:RA}(tamper{TCP:chksum:corrupt},))-| \/TCB Teardowns are not new: they were demonstrated almost a decade ago by Khattak et al., and Geneva has discovered Teardown attacks repeatedly in the past against the GFW.
Surprisingly, this strategy also can be induced from the server-side. During the three-way handshake, the server can send a
SYN+ACKpacket with a corrupt acknowledgement number, thereby inducing the client to send aRST. This causes theRSTto have an incorrect sequence number (and an acknowledgement number of 0, but it still is sufficient to cause a TCB Teardown.Strategy 4:
FIN+SYNThe next strategy appears to be another desychronization attack, but via a different attack vector. In this strategy, the client (or the server) sends a packet with the
FINandSYNflags both set during the three-way handshake. For the client, in Geneva's syntax:[TCP:flags:A]-duplicate(tamper{TCP:flags:replace:FS},)-| \/For the server, in Geneva's syntax:[TCP:flags:SA]-duplicate(tamper{TCP:flags:replace:FS},)-| \/In the past, we've found the GFW against other protocols has special handling for
FINpackets when it comes to resynchronization. In this case, it looks like the presence of theFINcauses the GFW to immediately resynchronize, but the presence of theSYNcauses it to think the actual seqno is+1from the actual value, making the GFW off by 1 from the real connection.We tested this hypothesis by incrementing the sequence number of the actual request by 1 while this strategy was running, and saw that the client got censored.
From the server-side, the
FINflag is not required for this strategy to work.Strategy 5: TCB Turnaround
The TCB Turnaround strategy is simple: before the client initiates the three-way handshake, it first sends a
SYN+ACKpacket to the server. TheSYN+ACKcauses the GFW to confuse the roles of the client and server, thereby allowing the client to communicate unimpeded. TCB Turnaround attacks still work in Kazakhstan, but turnaround attacks do not work against the GFW for any other protocols.In Geneva's syntax:
[TCP:flags:S]-duplicate(tamper{TCP:flags:replace:SA},)-| \/This strategy is client-only, since by the time the
SYNpacket arrives at the server, the censor already knows which side is the client.Strategy 6: TCB Desynchronization
Finally, Geneva identified simple payload-based TCB desynchronization. From the client, injecting a packet with a payload and a broken checksum is sufficient to desynchronize the GFW from the connection. Geneva has identified these in the past against the GFW's censorship of other protocols as well.
In Geneva's syntax:
[TCP:flags:A]-duplicate(tamper{TCP:load:replace:AAAAAAAAAA}(tamper{TCP:chksum:corrupt},),)-|This strategy cannot be used from the server-side.
Summary on Circumvention Strategies
In total, we have discovered 6 strategies that work from the client-side, and 4 that work from the server-side. Each of these works with near 100% reliability, and can be used to evade the ESNI censorship. Unfortunately, these specific strategies may not be a long-term solution: as the cat and mouse game progresses, the Great Firewall will likely to continue to improve its censorship capabilities.
Unresolved Questions
It is not yet clear why we observe different durations of residual censorship from different vantage points. As with all such research, it is also possible that there are some regions of China that are affected in different ways than our vantage points. If you observe different behavior or that some of our evasion strategies do not work, please feel free to contact us!
Thanks
We want to thank all anonymous reviewers who offered us valuable and immediate questions, feedback and suggestions on the riseup pad. These comments guided us to prioritize the questions that interest the community the most; and thus greatly accelerated our research.
We are also thankful to the OONI and OTF community for all of their support.
Contacts
Geneva team:
GFW Report: