Open wkrp opened 3 years ago
@censoredplanet, @reethikar, @ramakrishnansr: I imagine you have have evidence for DNS injection of twitter.com in Turkmenistan in Satellite data. You may also know other names that get injected. Is it convenient for you to check? Or, how would someone go about looking for country-specific injections in the raw data?
Yes, it is possible that we observe this phenomenon. @eltsai is currently working with @censoredplanet Satellite data. Elisa, can you please provide some information on whether we see this injection?
@wkrp thank you for sharing this! From the Satellite scan, we saw evidence of DNS injection for twitter.com
, as well as of some other domains listed below, all with injected IP 127.0.0.1
:
adultfriendfinder.com
alt.com
anonymouse.org
aparat.com
avaaz.org
bbc.com
blogspot.com
boingboing.net
bongacams.com
brave.com
bravotube.net
cbsnews.com
cloudflare-dns.com
community.livejournal.com
deviantart.com
digikala.com
discord.gg
divar.ir
dns.adguard.com
doh.centraleu.pi-dns.com
doh.dns.sb
doh.opendns.com
download.cnet.com
duckduckgo.com
dw.com
elpais.com
etsy.com
family.cloudflare-dns.com
freedns.afraid.org
goodreads.com
groups.google.com
hangouts.google.com
hola.org
hurriyet.com.tr
imdb.com
indianexpress.com
istockphoto.com
kizlarsoruyor.com
lenta.ru
line.me
livejasmin.com
livejournal.com
mk.ru
mozilla.cloudflare-dns.com
msn.com
mynet.com
netflix.com
nur.kz
ocsp.int-x3.letsencrypt.org
openvpn.net
pastebin.com
pornhub.com
protonmail.com
proxify.com
riseup.net
rsf.org
safervpn.com
scontent-ams4-1.cdninstagram.com
securevpn.com
shutterstock.com
slate.com
soundcloud.com
stackexchange.com
static.xx.fbcdn.net
strongvpn.com
surfshark.com
t.co
tamtam.chat
tandfonline.com
telegra.ph
telegram.me
telegram.org
tiktok.com
tinyurl.com
twitter.com
ultrasurf.us
unsplash.com
varzesh3.com
vk.com
wetransfer.com
www.alarabiya.net
www.amnesty.org
www.anonymizer.com
www.anonymizer.ru
www.bbc.com
www.betternet.co
www.brookings.edu
www.cbsnews.com
www.change.org
www.cpj.org
www.dailymotion.com
www.dropbox.com
www.dw.com
www.f-secure.com
www.facebook.com
www.gaystarnews.com
www.goodreads.com
www.gotgayporn.com
www.hidemyass.com
www.hotspotshield.com
www.hrw.org
www.imdb.com
www.ipvanish.com
www.last.fm
www.leaseweb.com
www.lemonde.fr
www.linkedin.com
www.livejournal.com
www.martus.org
www.megaproxy.com
www.messenger.com
www.msn.com
www.no-porn.com
www.opendns.com
www.opensocietyfoundations.org
www.patreon.com
www.pornhub.com
www.privateinternetaccess.com
www.rferl.org
www.sex.com
www.snapchat.com
www.tiktok.com
www.torproject.org
www.transparency.org
www.weforum.org
www.whatsapp.com
www.xvideos.com
www.yelp.com
www.youporn.com
www.youtube.com
xvideos.com
yelp.com
yenisafak.com
youporn.com
youtube.com
Going through the block list above, one domain ocsp.int-x3.letsencrypt.org
stands out: it belongs to an ocsp server operated by let's encrypt. I confirmed this domain received injection when querying Turkmenistan hosts. However, there are three other issuers that (seemingly) perform the same functions: ocsp.int-x1.letsencrypt.org
, ocsp.int-x2.letsencrypt.org
, ocsp.int-x4.letsencrypt.org
, all of which are NOT injected with false response.
This shares a high similarity with a previous report about users from China unable to resolve x3 while x1, x2, x4 are not affected. There are multiple posts like this and all report that x3 is blocked in China (earliest post goes back to Jan 2020). Some user claims that they ran catchpoint test in China and that x3's availability is only 10% of that of x1, but unfortunately details are lost.
On a side note, in that report, users also speculate about reasons why only x3 is blocked. If the block is intentional, GFW probably would block the other three issuers as well. The following is the bogus DNS response users from China were seeing:
;; ANSWER SECTION: ocsp.int-x3.letsencrypt.org. 5580 IN CNAME ocsp.int-x3.letsencrypt.org.edgesuite.net. ocsp.int-x3.letsencrypt.org.edgesuite.net. 7185 IN CNAME a771.dscq.akamai.net. a771.dscq.akamai.net. 147 IN A 31.13.73.17
They suspect that the CNAME assigned by akamai might be "contaminated" due to being linked previously to a blocked website. They tested that dig a771.dscq.akamai.net
also gets the same bogus response, while CNAMEs assigned to x1,2,4 are not affected. It seems that let's encrypt is currently working on a solution for this.
This seems interesting to me because we are observing exactly the same policy of DNS interference in Turkmenistan, including x1,2,4 not being affected and that 'a771.dscq.akamai.net' triggers interference. Are the censors from the two countries sharing block list in some way .. (?)
EDIT
On a closer look, the policy is slightly different between China & TM's DNS injector wrt ocsp.int-x3.letsencrypt.org
Domain | China Resolver Response | TM Resolver Response |
---|---|---|
ocsp.int-x{1,2,4}.letsencrypt.org | Real | Real |
ocsp.int-x3.letsencrypt.org | Bogus | Bogus |
ocsp.int-x{1,2,4}.letsencrypt.org.edgesuite.net | Real | Real |
ocsp.int-x3.letsencrypt.org.edgesuite.net | Bogus | Bogus |
a1914.dscq.akamai.net (CNAME for x1,2,4) | Real | Bogus |
a771.dscq.akamai.net (CNAME for x3) | Bogus | Bogus |
*.akamai.net* seems always to trigger injection on the TM resolver tested. So they do have different policies.
Quote @diwenx: Seems like for twitter.com
, DNS over UDP would trigger the 127.0.0.1
injected response, yet DNS over TCP would not:
$dig @$TARGET +noedns twitter.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 42542
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;twitter.com. IN A
;; ANSWER SECTION:
twitter.com. 300 IN A 127.0.0.1
$ dig @$TARGET +noedns twitter.com +tcp
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1338
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 10, ADDITIONAL: 0
;; QUESTION SECTION:
;twitter.com. IN A
;; ANSWER SECTION:
twitter.com. 1800 IN A 104.244.42.65
twitter.com. 1800 IN A 104.244.42.129
;; AUTHORITY SECTION:
twitter.com. 13999 IN NS c.r06.twtrdns.net.
twitter.com. 13999 IN NS ns4.p34.dynect.net.
twitter.com. 13999 IN NS ns1.p34.dynect.net.
twitter.com. 13999 IN NS ns3.p34.dynect.net.
twitter.com. 13999 IN NS a.r06.twtrdns.net.
twitter.com. 13999 IN NS b.r06.twtrdns.net.
twitter.com. 13999 IN NS d01-01.ns.twtrdns.net.
twitter.com. 13999 IN NS ns2.p34.dynect.net.
twitter.com. 13999 IN NS d.r06.twtrdns.net.
twitter.com. 13999 IN NS d01-02.ns.twtrdns.net.
;; Query time: 2296 msec
;; SERVER: 95.85.115.190#53(95.85.115.190)
;; WHEN: Fri Aug 20 23:38:16 EDT 2021
;; MSG SIZE rcvd: 268
From the Satellite scan, we saw evidence of DNS injection for
twitter.com
, as well as of some other domains listed below, all with injected IP127.0.0.1
:
Thanks for checking! It's wonderful that we now live in a world where we can answer questions like these quickly, because the necessary data already exist. This is thanks to you and all who have worked to build capable censorship measurement platforms.
An interesting feature of the DNS injection list I want to call out: many of the names are DNS over HTTP or DNS over TLS servers.
dns.adguard.com
freedns.afraid.org
cloudflare-dns.com
family.cloudflare-dns.com
mozilla.cloudflare-dns.com
doh.dns.sb
doh.opendns.com
doh.centraleu.pi-dns.com
Their presence on the list looks like an attempt to block encrypted DNS, which is not vulnerable to injected responses the way plaintext DNS is. For example, mozilla.cloudflare-dns.com
is one of the providers available by default in Firefox. It is possible to hard-configure an IP address for the DoH resolver in Firefox (set network.trr.bootstrapAddress
), but if you do not, then Firefox looks up the address of the DoH resolver using plaintext DNS, which the 127.0.0.1 injection would prevent.
I tested the domains of a handful of the DoH resolvers listed at the curl wiki, and most of them also experience injection:
$ for name in dns.google doh.xfinity.com dns.blokada.org doh.42l.fr doh.crypto.sz rethinkdns.com dns.this.web.id; do printf "%s\t" $name; dig @$TARGET +noedns +short $name; done
dns.google 127.0.0.1
doh.xfinity.com 127.0.0.1
dns.blokada.org 127.0.0.1
doh.42l.fr 127.0.0.1
doh.crypto.sz 127.0.0.1
rethinkdns.com 127.0.0.1
dns.this.web.id ;; connection timed out; no servers could be reached
A further interesting point about DoH is that there is no injection for the canary domain use-application-dns.net
. If a query for that name were to receive an A response, it would disable default DoH in Firefox.
$ dig @$TARGET +noedns use-application-dns.net
; <<>> DiG 9.11.5-P4-5.1+deb10u5-Debian <<>> @95.85.120.6 +noedns use-application-dns.net
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached
Even if you did know an IP address for a DoH resolver in advance, you likely still would not be able to use it. I didn't mention yet that RST injection in response to TLS SNI in Turkmenistan is also externally measurable:
$ curl --connect-to ::telecom.tm: https://cloudflare-dns.com/ -D -
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to cloudflare-dns.com:443
The SSL_ERROR_SYSCALL
message is because the TCP connection was reset. If there were no RST injection, we would instead expect a certificate mismatch error, as the telecom.tm web server serves its own certificate:
$ curl --connect-to ::telecom.tm: https://wikipedia.org/ -D -
curl: (60) SSL: no alternative certificate subject name matches target host name 'wikipedia.org'
The SNI blocklist is not the same as the DNS blocklist, though. For example, twitter.com is on both lists, but facebook.com is only on the SNI blocklist, and msn.com is only on the DNS blocklist.
Seems like for twitter.com, DNS over UDP would trigger the 127.0.0.1 injected response, yet DNS over TCP would not:
A quick test also shows that there is no injection with UDP over IPv6:
$ TARGET6=2a05:2180::1234 # the prefix comes from https://bgp.he.net/AS20661#_prefixes6
$ dig @$TARGET6 +noedns twitter.com
; <<>> DiG 9.11.5-P4-5.1+deb10u5-Debian <<>> @2a05:2180::1234 +noedns twitter.com
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached
Going through the block list above, one domain
ocsp.int-x3.letsencrypt.org
stands out: it belongs to an ocsp server operated by let's encrypt. I confirmed this domain received injection when querying Turkmenistan hosts. However, there are three other issuers that (seemingly) perform the same functions:ocsp.int-x1.letsencrypt.org
,ocsp.int-x2.letsencrypt.org
,ocsp.int-x4.letsencrypt.org
, all of which are NOT injected with false response.
That's a great find—thanks for finding this connection.
I tried making a CNAME on my own domain, pointing to a771.dscq.akamai.net
, to see if it would get injection the same way ocsp.int-x3.letsencrypt.org
does. It did not:
$ dig @8.8.8.8 akacname.mydomain.example
; <<>> DiG 9.11.5-P4-5.1+deb10u5-Debian <<>> @8.8.8.8 akacname.mydomain.example
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 55810
;; flags: qr rd ra; QUERY: 1, ANSWER: 7, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;akacname.mydomain.example. IN A
;; ANSWER SECTION:
akacname.mydomain.example. 1798 IN CNAME a771.dscq.akamai.net.
a771.dscq.akamai.net. 19 IN A 23.38.189.195
a771.dscq.akamai.net. 19 IN A 23.38.189.233
a771.dscq.akamai.net. 19 IN A 23.38.189.160
a771.dscq.akamai.net. 19 IN A 23.38.189.216
a771.dscq.akamai.net. 19 IN A 23.38.189.144
a771.dscq.akamai.net. 19 IN A 23.38.189.168
;; Query time: 54 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sat Aug 21 16:18:15 MDT 2021
;; MSG SIZE rcvd: 186
$ dig @$TARGET +noedns akacname.mydomain.example
; <<>> DiG 9.11.5-P4-5.1+deb10u5-Debian <<>> @95.85.120.6 +noedns akacname.mydomain.example
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached
.akamai.net seems always to trigger injection on the TM resolver tested. So they do have different policies.
I confirm that there seems to be a DNS injection rule for *.akamai.net*
.
I may have an explanation for the blocking of *.akamai.net
. Back in the day, it was possible to do a sort of proto–domain fronting using a special URL format that encoded the hostname in the URL path:
A typical embedded object URL such as http[]()://www.foo.com/images/logo.gif would be transformed into the following ARL: http[]()://a836.g.akamaitech.net/7/836/123/e358f5db0045e9/www.foo.com/images/logo.gif
The Akamai domain ensures that requests for akamaized content travel directly from the user to an Akamai server, completely avoiding the object’s home site. With rare exception, this field will be set to g.akamai.net.
As I understand it, this encoding scheme existed for compatibility with pre-HTTP/1.1 clients that do not send the Host header. I don't know whether it still works that way.
HTTPS URLs under akamai.net
were in fact used for hosting blocking-resistant mirrors:
On that note, it's worth looking at what GreatFire.org is doing for some of their mirror sites ... The long URL is an explicit form of what normally happens implicitly through SNI at the Akamai CDN. The important thing is that all the blockable content is encrypted in the path component. The censor only gets to see the domain name a248.e.akamai.net, which is some kind of magic Akamai HTTPS domain that's used for tons of stuff.
Possibly as a result of being used for circumvention purposes, a248.e.akamai.net become blocked by DNS injection in China in September 2014:
A very similar event happened at the end of September on Akamai, but it didn't get as much attention. The special domain a248.e.akamai.net is roughly the equivalent of edgecastcdn.net, was also used for mirror hosting, and it was also blocked by DNS poisoning. https://en.greatfire.org/https/a248.e.akamai.net The few web sites that happened to hardcode a248.e.akamai.net would have been broken, but that's not the common way to use Akamai. The common way is to use your own CNAME, and let Akamai direct the request according to the Host header (which is exactly what we take advantage of in domain fronting). GFW didn't block everything on Akamai (which would have truly enormous collateral damage), they only blocked one specific name.
Even if you did know an IP address for a DoH resolver in advance, you likely still would not be able to use it. I didn't mention yet that RST injection in response to TLS SNI in Turkmenistan is also externally measurable:
We confirm this from the Hyperquack HTTP and HTTPS data. From HTTPS data in Turkmenistan during 2020, we see 82 domains are experiencing TCP reset after Sep 13 2020 (a lot of domains owned by Google):
addons.mozilla.org
allo.google.com
amazon.com.au
amazon.com.br
amazon.com.mx
apple.com
ar.m.wikipedia.org
ar.wikipedia.org
bing.com
ca.wikipedia.org
de.wikipedia.org
docs.google.com
dw.com
en.m.wikipedia.org
en.wikipedia.org
encrypted.google.com
es.m.wikipedia.org
es.wikipedia.org
fr.wikipedia.org
github.com
google.com
google.com.ar
google.com.au
google.com.br
google.com.co
google.com.eg
google.com.hk
google.com.mx
google.com.my
google.com.pe
google.com.pk
google.com.sa
google.com.sg
google.com.tr
google.com.tw
google.com.ua
google.com.vn
googlevideo.com
groups.google.com
hangouts.google.com
imgur.com
istockphoto.com
it.wikipedia.org
ja.wikipedia.org
lalgbtcenter.org
mail.yandex.ru
microsoft.com
namasha.com
netflix.com
news.google.com
ocsp.int-x3.letsencrypt.org
paypal.com
picasa.google.com
protonvpn.com
pt.m.wikipedia.org
ru.wikipedia.org
shutterstock.com
sites.google.com
taobao.com
tinyurl.com
translate.google.com
twitch.tv
video.google.com
weather.com
wikipedia.org
www.alarabiya.net
www.baidu.com
www.dw.com
www.google.com
www.gotgayporn.com
www.opendns.com
www.pandora.com
www.paypal.com
www.towleroad.com
www.truecaller.com
www.twitch.tv
www.wikipedia.org
www.xvideos.com
www.youtube.com
yandex.com
yandex.ru
zh.wikipedia.org
From the Hyperquack HTTP data, we see 45 domains are almost blocked all year long, except a temporal removal in a single netblock (217.174.224.0/20
, "TURKMENTELECOM-NET") during September.
addons.mozilla.org
allo.google.com
amazon.com.au
amazon.com.br
apple.com
beeg.com
bing.com
docs.google.com
encrypted.google.com
github.com
google.com
google.com.ar
google.com.au
google.com.br
google.com.co
google.com.eg
google.com.hk
google.com.mx
google.com.my
google.com.pe
google.com.pk
google.com.sa
google.com.tw
google.com.ua
google.com.vn
hangouts.google.com
mail.yandex.ru
mozilla.org
netflix.com
news.google.com
picasa.google.com
protonvpn.com
sites.google.com
strongvpn.com
translate.google.com
video.google.com
weather.com
www.apple.com
www.google.com
www.mozilla.org
www.netflix.com
www.purevpn.com
www.transferbigfiles.com
yandex.com
yandex.ru
Interestingly, I did not see a sudden drop in traffic in the Google Transparency Report in Turkmenistan, not for Web Search at least. I can see a decrease in the traffic for Youtube and Google Sites, not sure if this is because of the blocking.
We confirm this from the Hyperquack HTTP and HTTPS data.
Thanks for checking. I can reproduce your findings. Here are some more characteristics of the HTTP and SNI injection:
You can demonstrate the injection easily using curl or OpenSSL s_client. First, get an IP address in Turkmenistan:
$ dig +short telecom.tm
95.85.120.6
$ TARGET=95.85.120.6
The host happens to be an HTTP and HTTPS server. In the case of no injection, we expect to see an HTTP response or a TLS Server Hello with the server's normal certificate:
$ curl --connect-to ::$TARGET: http://wikipedia.org/ -D -
HTTP/1.1 301 Moved Permanently
...
$ curl --connect-to ::$TARGET: https://wikipedia.org/ -D -
curl: (60) SSL: no alternative certificate subject name matches target host name 'wikipedia.org'
...
$ openssl s_client -connect $TARGET:443 -servername wikipedia.org
...
subject=CN = *.telecom.tm
...
In contrast, when there is an injection, we instead get an error showing that a RST was received:
$ curl --connect-to ::$TARGET: http://twitter.com/ -D -
curl: (56) Recv failure: Connection reset by peer
$ curl --connect-to ::$TARGET: https://twitter.com/ -D -
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to twitter.com:443
$ openssl s_client -connect $TARGET:443 -servername twitter.com
CONNECTED(00000003)
write:errno=104
...
Injection occurs even if the client sends HTTP or HTTPS to the "wrong" port out of 80 and 443, but not on other ports:
$ curl --connect-to ::$TARGET: http://twitter.com:443/ -D -
curl: (56) Recv failure: Connection reset by peer
$ curl --connect-to ::$TARGET: https://twitter.com:80/ -D -
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to twitter.com:80
$ curl --connect-to ::$TARGET: --connect-timeout 5 http://twitter.com:81/ -D -
curl: (28) Connection timed out after 5001 milliseconds
$ curl --connect-to ::$TARGET: --connect-timeout 5 https://twitter.com:444/ -D -
curl: (28) Connection timed out after 5001 milliseconds
In my tests, packets that are actually from 95.85.120.6 (such as the SYN/ACK) have TTL 47, while the RSTs have TTL 113.
The matching rules are not the same for all three forms of injection. You can see that in the differing lengths of the lists @eltsai posted. For example:
name | DNS | HTTP | SNI |
---|---|---|---|
msn.com | INJECT | RST | ok |
example.com | ok | ok | RST |
x.akamai.net | INJECT | ok | RST |
But even for domains that are common to two or more lists, wildcard rules may differ. For example, the HTTP rule matches *msn.com*
, but the DNS rule matches only msn.com
exactly.
name | DNS | HTTP | SNI |
---|---|---|---|
msn.com | INJECT | RST | ok |
xmsn.com | ok | RST | ok |
x.msn.com | ok | RST | ok |
msn.comx | ok | RST | ok |
msn.com.x | ok | RST | ok |
$ for n in msn.com; do for name in $n x$n x.$n ${n}x $n.x; do echo; echo == $name; dig @$TARGET +noedns +short +timeout=5 $name; curl --connect-to ::$TARGET: http://$name/; curl --connect-to ::$TARGET: https://$name/; done; done
From the Hyperquack HTTP data, we see 45 domains are almost blocked all year long...
protonvpn.com strongvpn.com www.purevpn.com
@ValdikSS reports that any HTTP request with a Host header that contains the string "vpn" gets an immediate RST. The same rule does not exist for HTTPS SNI. Only lowercase "vpn" is matched.
$ curl --connect-to ::95.85.120.6: --connect-timeout 5 http://example.com/ -D -
HTTP/1.1 301 Moved Permanently
$ curl --connect-to ::95.85.120.6: --connect-timeout 5 http://examvpnle.com/ -D -
curl: (56) Recv failure: Connection reset by peer
$ curl --connect-to ::95.85.120.6: --connect-timeout 5 http://EXAMVPNLE.COM/ -D -
HTTP/1.1 301 Moved Permanently
@wkrp First of all, I would like to thank you all for your work on censorship analysis.
I am a resident of Turkmenistan and after reading this I was glad that someone was analyzing all this. I understand that several blocking methods are used, but could you tell about the methods for bypassing them? For its part, I am ready to provide any data. Thanks
I understand that several blocking methods are used, but could you tell about the methods for bypassing them?
@cenZnah I don't know what works best in Turkmenistan currently. NTC has a sub-forum dedicated to Turkemenistan; you can check some threads there:
https://ntc.party/c/internet-censorship-all-around-the-world/turkmenistan/17
E.g.
There's a good chance that Tor Browser with a private obfs4 bridge will work. You can request a private bridge by emailing frontdesk@torproject.org: see instructions.
According to the Psiphon dashboard, there are over 2K daily unique users in Turkmenistan in the last week, so it is worth trying Psiphon.
For its part, I am ready to provide any data.
Thank you for the offer of help. It has been hard to make contact with users in Turkmenistan. Here are some threads with requests for assistance:
Let me know if you have difficulty accessing any of these links. We can also establish some efficient communications channels for faster feedback if necessary.
Where response construction goes wrong
The simplistic technique of appending a fixed resource record to an observed query only works when the query does not contain any resource records after the Question section. If it does, then the injector will simply reflect those resource records back to the requester, along with its additional suffix. The requester will interpret the first of its own resource records in the response as belonging to the Answer section, and the injector's added suffix will appear as unused trailing bytes.
This can be easily demonstrated by running dig without the
+noedns
option. By default, dig sends queries with EDNS, which means they have a non-empty Additional section. The injector appends what it intends to be the response's Answer section without first removing the query's Additional section. The requester sees its own Additional section as being the Answer section.
I tried the Turkmenistan DNS injector again today, and it looks like it has changed in this regard. It no longer keeps data from the query after the first Question. That means it no longer produces malformed responses to EDNS queries.
See how the FFFFFFFF
is no longer included in the response:
$ (printf '\x12\x34\x01\x20\x00\x01\x00\x00\x00\x00\x00\x00\x07twitter\x03com\x00\x00\x01\x00\x01\xff\xff\xff\xff'; sleep 1) | ncat --udp $TARGET 53 -x /dev/stderr >/dev/null [0000] 12 34 01 20 00 01 00 00 00 00 00 00 07 74 77 69 .4. .... .....twi [0010] 74 74 65 72 03 63 6F 6D 00 00 01 00 01 FF FF FF tter.com ........ [0020] FF . [0000] 12 34 81 80 00 01 00 01 00 00 00 00 07 74 77 69 .4...... .....twi [0010] 74 74 65 72 03 63 6F 6D 00 00 01 00 01 FF FF FF tter.com ........ [0020] FF C0 0C 00 01 00 01 00 00 01 2C 00 04 7F 00 00 ........ ..,..... [0030] 01 .
2022-11-17
$ (printf '\x12\x34\x01\x20\x00\x01\x00\x00\x00\x00\x00\x00\x07twitter\x03com\x00\x00\x01\x00\x01\xff\xff\xff\xff'; sleep 1) | ncat --udp $TARGET 53 -x /dev/stderr >/dev/null [0000] 12 34 01 20 00 01 00 00 00 00 00 00 07 74 77 69 .4. .... .....twi [0010] 74 74 65 72 03 63 6F 6D 00 00 01 00 01 FF FF FF tter.com ........ [0020] FF . [0000] 12 34 81 80 00 01 00 01 00 00 00 00 07 74 77 69 .4...... .....twi [0010] 74 74 65 72 03 63 6F 6D 00 00 01 00 01 C0 0C 00 tter.com ........ [0020] 01 00 01 00 00 01 2C 00 04 7F 00 00 01 ......,. .....
The Question section may contain more than one entry, and the
twitter.com
entry does not have to be the first one. Notice that QDCOUNT=0001
in the response, not0002
as it was in the query.
A weird consequence is that a query that has 2 Questions, where only the 2nd Question is filtered, will have a response that refers to only the first, non-filtered name.
$ (printf '\x12\x34\x01\x20\x00\x02\x00\x00\x00\x00\x00\x00\x03foo\x03com\x00\x00\x01\x00\x01\x07twitter\x03com\x00\x00\x01\x00\x01'; sleep 1) | ncat --udp $TARGET 53 -x /dev/stderr >/dev/null [0000] 12 34 01 20 00 02 00 00 00 00 00 00 03 66 6F 6F .4. .... .....foo [0010] 03 63 6F 6D 00 00 01 00 01 07 74 77 69 74 74 65 .com.... ..twitte [0020] 72 03 63 6F 6D 00 00 01 00 01 r.com... .. [0000] 12 34 81 80 00 01 00 01 00 00 00 00 03 66 6F 6F .4...... .....foo [0010] 03 63 6F 6D 00 00 01 00 01 07 74 77 69 74 74 65 .com.... ..twitte [0020] 72 03 63 6F 6D 00 00 01 00 01 C0 0C 00 01 00 01 r.com... ........ [0030] 00 00 01 2C 00 04 7F 00 00 01 ...,.... ..
2022-11-17
$ (printf '\x12\x34\x01\x20\x00\x02\x00\x00\x00\x00\x00\x00\x03foo\x03com\x00\x00\x01\x00\x01\x07twitter\x03com\x00\x00\x01\x00\x01'; sleep 1) | ncat --udp $TARGET 53 -x /dev/stderr >/dev/null [0000] 12 34 01 20 00 02 00 00 00 00 00 00 03 66 6F 6F .4. .... .....foo [0010] 03 63 6F 6D 00 00 01 00 01 07 74 77 69 74 74 65 .com.... ..twitte [0020] 72 03 63 6F 6D 00 00 01 00 01 r.com... .. [0000] 12 34 81 80 00 01 00 01 00 00 00 00 03 66 6F 6F .4...... .....foo [0010] 03 63 6F 6D 00 00 01 00 01 C0 0C 00 01 00 01 00 .com.... ........ [0020] 00 01 2C 00 04 7F 00 00 01 ..,..... .
The injector's DNS parser does not seem to fully understand compression pointers, though. Indeed, the appearance of a compression pointer seems to halt interpretation of a name. If you replace the
00
null label at the end of the QNAME with either a forward or a backward pointer to some other00
byte in the message, you get an injection. But if you replace thecom
label with a pointer to acom
label elsewhere in the packet, you do not get an injection—even though such queries work fine with normal resolvers like 1.1.1.1:
Replacing the 00
name terminator label with a compression pointer no longer gives me an injection.
$ (printf '\x12\x34\x01\x20\x00\x01\x00\x00\x00\x00\x00\x00\x07twitter\x03com\xc0\x04\x00\x01\x00\x01'; sleep 1) | ncat --udp $TARGET 53 -x /dev/stderr >/dev/null [0000] 12 34 01 20 00 01 00 00 00 00 00 00 07 74 77 69 .4. .... .....twi [0010] 74 74 65 72 03 63 6F 6D C0 04 00 01 00 01 tter.com ...... [0000] 12 34 81 80 00 01 00 01 00 00 00 00 07 74 77 69 .4...... .....twi [0010] 74 74 65 72 03 63 6F 6D C0 04 00 01 00 01 C0 0C tter.com ........ [0020] 00 01 00 01 00 00 01 2C 00 04 7F 00 00 01 ......., ......
2022-11-17
$ (printf '\x12\x34\x01\x20\x00\x01\x00\x00\x00\x00\x00\x00\x07twitter\x03com\xc0\x04\x00\x01\x00\x01'; sleep 1) | ncat --udp $TARGET 53 -x /dev/stderr >/dev/null [0000] 12 34 01 20 00 01 00 00 00 00 00 00 07 74 77 69 .4. .... .....twi [0010] 74 74 65 72 03 63 6F 6D C0 04 00 01 00 01 tter.com ......
Not sure that's valuable, but related to the topic - found DNS server in Turkmenistan which could be used to check for DNS injection: 217.174.239.141
Hi @starlancer1 , thanks for sharing!
You probably already knew this, but we are sharing it here just in case. When measuring the DNS injections, it is sometimes even more desirable to send DNS queries to a non-DNS server/resolver for testing. This way, you can guarantee that the DNS responses you received are injected by some middle boxes, rather than some real DNS servers/resolvers. This measurement design gives you zero false positive in detection.
For example, in your case, you need to somehow validate the responses from 217.174.239.141
to decide if the response of tdh.gov.tm
is "fine" or not. Sometimes this process can be as easy as checking if the answer is 127.0.0.1
or not, but sometimes it can be more complex than this as the fingerprints can change over time.
But if you send queries to a non-DNS server/resolver like 217.174.239.140
, you will certainly know tdh.gov.tm
is not censored because you never receive any responses after multiple queries (your queries will just time out).
@gfw-report actually I did not, so thanks for sharing - all clear now!
While investigating a recent change in Tor metrics in Turkmenistan, I found a DNS injection that can be triggered by sending DNS queries into the country. Query for any name matching the pattern
*twitter.com*
will receive an injected response, specifically an A record pointing to the IP address 127.0.0.1.I don't doubt that someone local to the region has already discovered this, but I did not find anything in a quick web search, so I'm documenting what I found here.
I only tested the outside-in direction. I did not actually verify that the injection occurs from a vantage inside Turkmenistan, but I would be surprised if that were not the case.
Summary
*twitter.com*
.twitter.com
causes injection, as do names with prefixes and suffixes:x.twitter.com
,xtwitter.com
,twitter.com.x
,twitter.comx
.c00c000100010000012c00047f000001
.Demonstration using dig
We will need an IP address in Turkmenistan to send queries to:
Issue a query, using the Turkmenistan IP address as the resolver:
Look at
status: NOERROR
andQUERY: 1, ANSWER: 1
. This means the query received a response with one answer. TheANSWER SECTION
shows that according to the injected response, the name resolves to 127.0.0.1.Analysis of payloads
Let's take a look at what happens at the DNS protocol level. From a packet capture, we see that the 29 bytes of the query message are:
12 34
01 20
00 01
00 00
00 00
00 00
07 74 77 69 74 74 65 72
07
is the length of the label, and74 77 69 74 74 65 72
spellstwitter
.03 63 6f 6d
03
is the length and63 6f 6d
spellscom
.00
00 01
00 01
The 45 bytes of the injected response are almost the same. I've marked the changed or added bytes with
()
:The response is the same as the query, except for these changes:
0120
to8180
(now with QR=1, indicating a response message).0000
to0001
(indicating a single resource record in the Answer section),c00c000100010000012c00047f000001
.Let's analyze the suffix, which syntactically forms a resource record in the response's Answer section.
c0 0c
c00c
is a compression pointer that points to byte offset 12, the first name in the Question section; i.e., the name that appeared in the query.00 01
00 01
00 00 01 2c
00 04
7f 00 00 01
Where response construction goes wrong
The simplistic technique of appending a fixed resource record to an observed query only works when the query does not contain any resource records after the Question section. If it does, then the injector will simply reflect those resource records back to the requester, along with its additional suffix. The requester will interpret the first of its own resource records in the response as belonging to the Answer section, and the injector's added suffix will appear as unused trailing bytes.
This can be easily demonstrated by running dig without the
+noedns
option. By default, dig sends queries with EDNS, which means they have a non-empty Additional section. The injector appends what it intends to be the response's Answer section without first removing the query's Additional section. The requester sees its own Additional section as being the Answer section.Note the EDNS
OPT PSEUDOSECTION
andWARNING: Message has 16 extra bytes at end
.Despite the response message being syntactically broken, it likely still has the intended effect of preventing name resolution.
Further experiments with netcat
For better control over the content of queries, we can construct our own UDP packets and send them using netcat. The
-x
option of Ncat is convenient for getting a hex dump. For example, here, the first two lines are the outgoing query and the next three lines are the incoming (injected) response.Queries sent to ports other then 53 (for example port 12345) also get injected. Even port 0 works.
If you set QR=1 (change FLAGS from
0120
to8120
), you do not get an injection.But, you can set bit in FLAGS other than QR (i.e. FLAGS=
7fff
), and you will get an injection. The response's FLAGS are overwritten to8180
as usual.Sending a query with QTYPE=AAAA (
00 1c
) still results in an injected A-type response.You can also freely change QCLASS (here
ff ff
).But, it seems that QCLASS≠IN only gets an injection on port 53! On other ports, it does not.
And if you truncate the query before QTYPE and QCLASS, you do not get an injection.
Any trailing bytes in the query (here
ff ff ff ff
) are retained, leading to a syntactically invalid response, as in the EDNS demonstration above.If you stick garbage before the Question section, you do not get an injection, which suggests there is an actual DNS parser in play, not just a simple byte pattern matcher.
Various name mutations show that the matching rule is probably
*twitter.com*
.twitter.com
may have an arbitrary prefix and suffix, even without an intervening name separator.twitter.com
witter.com
twitter.co
xtwitter.com
twitter.comx
x.twitter.com
twitter.com.x
You also get an injection if you represent the name
twitter.com
(invalidly) as a single 11-byte label (including the dot character in the middle), rather than as two labels of 7 and 3 bytes,twitter
andcom
. This suggests that the matching algorithm extracts the name from the query as a text string, then matches on the resulting string.The Question section may contain more than one entry, and the
twitter.com
entry does not have to be the first one. Notice that QDCOUNT=0001
in the response, not0002
as it was in the query.The injector's DNS parser does not seem to fully understand compression pointers, though. Indeed, the appearance of a compression pointer seems to halt interpretation of a name. If you replace the
00
null label at the end of the QNAME with either a forward or a backward pointer to some other00
byte in the message, you get an injection. But if you replace thecom
label with a pointer to acom
label elsewhere in the packet, you do not get an injection—even though such queries work fine with normal resolvers like 1.1.1.1: