The blocking of 1.1.1.1 in China, starting from 2023-10-01

gfw-report commented 9 months ago

There have been many reports of a blocking of 1.1.1.1 in China, starting from October 1, 2023.

As discussed in issues https://github.com/net4people/bbs/issues/285, China injected TCP RST packets to block 1.1.1.1:443 from September 5 to 20, 2023.

Major observations

Below is our observation from a VPS in Tencent Cloud Beijing (ASN AS45090) on October 1, 2023:

Different from @5e2t 's observation, we have not been able to observe the TCP RSTs on the 1.1.1.1:443 from our vantage point. In particular, we can successfully retrieve a complete webpage using curl -v https://1.1.1.1. This shows inconsistency of this new censorship incident across different geo-locations or ASes.
We observed that there was a chance that the TCP port 80 of 1.1.1.1 got injected with a "302 Moved Temporarily" or "301 Moved Permanently" message, attempting to redirect users to the National Anti-Fraud Center website (wiki).

Analysis on the injection to `1.1.1.1:80`

Here is one example when no injection happens:

ubuntu@VM-32-5-ubuntu:~$ curl -v http://1.1.1.1
*   Trying 1.1.1.1:80...
* TCP_NODELAY set
* Connected to 1.1.1.1 (1.1.1.1) port 80 (#0)
> GET / HTTP/1.1
> Host: 1.1.1.1
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 301 Moved Permanently
< Server: cloudflare
< Date: Sun, 01 Oct 2023 22:49:54 GMT
< Content-Type: text/html
< Content-Length: 167
< Connection: keep-alive
< Location: https://1.1.1.1/
< CF-RAY: **REDACTED**-SJC
<
<html>
<head><title>301 Moved Permanently</title></head>
<body>
<center><h1>301 Moved Permanently</h1></center>
<hr><center>cloudflare</center>
</body>
</html>
* Connection #0 to host 1.1.1.1 left intact

This is one example when the "302 Moved Temporarily" got injected.

ubuntu@VM-32-5-ubuntu:~$ curl -v http://1.1.1.1
*   Trying 1.1.1.1:80...
* TCP_NODELAY set
* Connected to 1.1.1.1 (1.1.1.1) port 80 (#0)
> GET / HTTP/1.1
> Host: 1.1.1.1
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 302 Moved Temporarily
< Connection: close
< Location: http://182.43.124.6/fzyujing?parameter2=REDACTED
<
* Closing connection 0

In particular, the redacted parameter in the output consists of 319 characters. Querying from the same vantage point across time, only the 129th to 150th characters (22 characters) and the 257th to 278th characters (22 characters) in the 319 character message got changed. It is still unclear to us what information got encoded in this parameter.

The real 301 Moved Permanently response from the 1.1.1.1 will eventually get to the client (but arrived later than the injected message), indicating the censor doesn't drop the real response from 1.1.1.1:80.

The ASN of the IP 182.43.124.6 that hosts the National Anti-Fraud Center website:

host	asn	asname	cc	registry
182.43.124.6	AS58519	CHINATELECOM-CTCLOUD Cloud Computing Corporation, CN	CN	apnic

5e2t commented 9 months ago

Screenshot from 2023-10-02 08-48-19

My location is Henan , but the RST I received is not from Henan GFW's censorship equipment. It's from ShangHai exit GFW's censorship equipment. because the latency between me and ShangHai exit is 28ms, traceroute also shows network path is through ShangHai exit.

5e2t commented 9 months ago

when I curl -v http://1.1.1.1 I received this in 30ms after HTTP GET sent.

HTTP/1.1 301 Moved Permanently Location: http://106.74.25.198/ Cache-Control: no-cache Content-Type: text/html Content-Length: 0

Screenshot from 2023-10-02 09-02-39

5e2t commented 9 months ago

content of http://106.74.25.198/ Screenshot from 2023-10-02 09-28-32

UjuiUjuMandan commented 9 months ago

content of http://106.74.25.198/

I really dialed the complaint number they provided, but no one answered (the first 2 numbers), and it "cannot go through" (the last one).

klzgrad commented 8 months ago

Injected HTTP/1.1 301 Moved Permanently packets have IP ids of 0x99d1, 0x99d2, 0x99d3, 0x99d4. Injected HTTP/1.1 302 Moved Temporarily packets have IP id of 0x4c57. They also have consistent TTLs.

GET requests with invalid Host fields, e.g. printf 'GET / HTTP/1.1\r\n\r\n' | nc -v 1.1.1.1 80 or printf 'GET / HTTP/1.1\r\nHost: \r\n\r\n' | nc -v 1.1.1.1 80 will yield a RST/ACK with the same IP ids and TTL as the injected HTTP 301 packets.

UjuiUjuMandan commented 8 months ago

The tampering was lifted since morning in my area.

gaukas commented 8 months ago

will yield a RST/ACK with the same IP ids and TTL as the injected HTTP 301 packets.

That's a really intriguing behavior. It could imply that these HTTP 301 responses were actually issued by a (group of) real HTTP server by redirecting/copying HTTP requests for it to handle, instead of hard-coded/crafted in advance.

It might be nice if it is possible to further exploit this mechanism in some ways? 😼

gfw-report commented 8 months ago

We conducted an ongoing experiment from a vantage point in Tencent Cloud Beijing (ASN AS45090). Specifically, we curl https://1.1.1.1 and curl http://1.1.1.1 every minute and capture the network traffic.

Below is an analyis based on the data we collected between Sunday, October 1, 2023 19:54 PM (Beijing Time, UTC+8) and Friday, October 6, 2023 2:43 PM (Beijing Time, UTC+8). In total, we made 6169 HTTP requests. We received 559 HTTP/1.1 301 Moved Permanently injected packets and 1760 HTTP/1.1 302 Moved Temporarily.

This table summarizes all possible values seen in each type of injected responses:

HTTP Status Code	301	302
Total Number of Injections	559	1760
Injection Ratio (over 6169 requests)	9.06%	28.5%
IP ID	0X99b3	0x4c57
IP TTL	251	251
IP Flags	0x0	0x0
TCP Flags	0x18 (PSH+ACK)	0x19 (PSH+ACK+FIN)
TCP Window Size	502	65535

In comparision with @klzgrad 's observation that:

Injected HTTP/1.1 301 Moved Permanently packets have IP ids of 0x99d1, 0x99d2, 0x99d3, 0x99d4. Injected HTTP/1.1 302 Moved Temporarily packets have IP id of 0x4c57.

We only observed one IP ID value 0x99b3 for the HTTP/1.1 301 Moved Permanently injection, which is dffierent from the four reported values.

They also have consistent TTLs.

We also observed consistent TTLs and its value is the same as the packets sent by the real 1.1.1.1 server.

The figure below shows the number of injections we received in each hour. We send around 60 requests in each hour and the average injection rate for 301 and 302 responses are only 9.06% and 28.5% respectively:

klzgrad commented 8 months ago

IP TTL 251 251

My HTTP 301 packets have TTLs of 249. But my HTTP 302 packets have TTLs of 119.

its value is the same as the packets sent by the real 1.1.1.1 server.

This must be a typo because the real 1.1.1.1 server returns TTLs of 64 - $hops, 55 in my testing.

were actually issued by a (group of) real HTTP server

Unlikely.

printf 'GET /\r\nHost: "\r\n\r\n' | nc -v 1.1.1.1 80: RST/ACK printf 'GET /\r\nHost: ""\r\n\r\n' | nc -v 1.1.1.1 80: Location: http://182.43.124.6/fzyujing? printf 'GET /\r\nHost: .\r\n\r\n' | nc -v 1.1.1.1 80: RST/ACK printf 'GET /\r\nHost: ..\r\n\r\n' | nc -v 1.1.1.1 80: Location: http://182.43.124.6/fzyujing?

gfw-report commented 8 months ago

My HTTP 301 packets have TTLs of 249. But my HTTP 302 packets have TTLs of 119. ... ... the real 1.1.1.1 server returns TTLs of 64 - $hops, 55 in my testing.

Thank you for pointing this out. It's very likely that there are some middleboxes, which are 4 hops away from our TecentCloud VPS (possibly as part of the TecentCloud infrastructure), rewriting all the IP TTL to 255 for packets coming to our VPS. The IP TTL of packets sent out from our VPS did not get rewritten.

We know this because we sent packets with initial IP TTL 64 between two hosts, and still observed packets coming to our VPS have the IP TTL 251.

In the pcap I captured, all 49,977 packets sent or got injected from 1.1.1.1:80 indeed have the same IP TTL 251:

tshark -Y "ip.src == 1.1.1.1 and tcp.srcport == 80" -r 1.1.1.1.pcap -Tfields -e ip.ttl | sort | uniq -c
  49977 251

including those 2,319 injected packets:

tshark -Y "ip.src == 1.1.1.1 and tcp.srcport == 80 and tcp.len != 0 and (not tcp.payload contains 43:46:2d:52:41:59)" -r 1.1.1.1.pcap -Tfields -e ip.ttl | sort | uniq -c
   2319 251

wkrp commented 8 months ago

The tampering was lifted since morning [2023-10-06] in my area.

I also received a separate report that the blocking stopped on 2023-10-04 about 07:30 UTC.

Here's an HTTP GET test from today, showing general reachability, I think: https://17ce.com/site/http/20231030_89f5d95076db11eeae206dcaa1273080:1.html (archive)

Screenshot of 17ce.com GET test for http://1.1.1.1, a map of China showing access times of between 0 and 3 seconds for most provinces.

I-I-IT commented 7 months ago

From Zhejiang Province : works seamlessly with wifi. Speed of 20-30 mbps. However, many websites block IPs from China.

net4people / bbs