search

AdguardTeam / AdGuardDNS

Public DNS resolver that protects you from ad trackers
https://adguard-dns.io/
GNU Affero General Public License v3.0
787 stars 61 forks source link

Akamai and EDNS Client Subnet issues #729

Open trmdi opened 9 months ago

trmdi commented 9 months ago

Issue Details

$ CLASS=CH RRTYPE="TXT" dnslookup id.server tls://dns.adguard.com                                      
dnslookup v1.9.1                                        
dnslookup result (elapsed 276.594529ms):
;; opcode: QUERY, status: NXDOMAIN, id: 60293           
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, 
AUTHORITY: 1, ADDITIONAL: 6                                            
;; QUESTION SECTION:                                    
;id.server.     CH       TXT                            ;; 
AUTHORITY SECTION:
.       1237    IN      SOA     a.root-servers.net. 
nstld.verisign-grs.com. 2024012202 1800 900 
604800 86400    
;; ADDITIONAL SECTION:
client-ip.adguard-dns.com.      3600    CH      TXT    "14.226.99.x"                                          
server-ip.adguard-dns.com.      3600    CH      TXT    "94.140.14.14"
country.adguard-dns.com.        3600    CH      TXT    "VN"
asn.adguard-dns.com.    3600    CH      TXT     "45899"
subdivision.adguard-dns.com.    3600    CH      TXT    "HP"                                                     resp.res- 
type.adguard-dns.com.  3600    CH      TXT    "normal"

Expected Behavior

When you are in the North of Vietnam, your ECS should be set to ones in the North, because the client should connect to CDNs in Hanoi/HCM/Hongkong for shorter distances and latency, instead of Singapore.

Actual Behavior

$ RRTYPE=TXT dnslookup o-o.myaddr.l.google.com tls://dns.adguard.com
dnslookup v1.9.1                                        
dnslookup result (elapsed 285.909138ms):
;; opcode: QUERY, status: NOERROR, id: 18760            
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:                                    ;o- 
o.myaddr.l.google.com.       IN       TXT

;; ANSWER SECTION:
o-o.myaddr.l.google.com.        60      IN      TXT    "edns0-client-subnet 222.255.238.0/24"
o-o.myaddr.l.google.com.        60      IN      TXT    "138.199.46.250"

$ ipinfo.sh 222.255.238.0           [11:14:28]
{
"ip": "222.255.238.0",
"hostname": "static.vnpt.vn",
"city": "Ho Chi Minh City",
"region": "Ho Chi Minh",
"country": "VN",
"loc": "10.8230,106.6296",
"org": "AS45899 VNPT Corp",
"postal": "71606",
"timezone": "Asia/Ho_Chi_Minh"
}
trmdi commented 9 months ago

From a screenshot in the video below, I see subdivision is applied to CN, IN... but not to VN. Is it right? https://github.com/AdguardTeam/AdGuardDNS/issues/659#issuecomment-1904324548 Screenshot_2024-01-23-11-38-36-158

trmdi commented 9 months ago

Adguard DNS:

$ dig starling-oversea.byteoversea.com @dns.adguard.com +short
starling-oversea.byteoversea.com.ttdns2.com.
starling-oversea.byteoversea.com.edgekey.net. 
e11942.a.akamaiedge.net.
23.215.7.32                                             23.215.7.28
23.215.7.19                                             23.215.7.16
23.215.7.22                                             23.215.7.23
23.215.7.20                                             23.215.7.25
23.215.7.18                                             

$ ping -c3 23.215.7.15             [11:49:13]
PING 23.215.7.15 (23.215.7.15) 56(84) bytes of data.
64 bytes from 23.215.7.15: icmp_seq=1 ttl=57 time=47.5 ms
64 bytes from 23.215.7.15: icmp_seq=2 ttl=57 time=46.1 ms
64 bytes from 23.215.7.15: icmp_seq=3 ttl=57 time=48.9 ms

--- 23.215.7.15 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 46.087/47.489/48.892/1.145 ms

Google DNS:

$ dig starling-oversea.byteoversea.com @dns.google +short
starling-oversea.byteoversea.com.ttdns2.com.
starling-oversea.byteoversea.com.edgekey.net.
e11942.a.akamaiedge.net.
113.171.10.152                                       113.171.10.144
113.171.10.162                                       113.171.10.160
113.171.10.155                                       113.171.10.154
113.171.10.146                                       113.171.10.139
113.171.10.145

$ ping -c3 113.171.10.155           [11:49:33]
PING 113.171.10.155 (113.171.10.155) 56(84) bytes of data.
64 bytes from 113.171.10.155: icmp_seq=1 ttl=58 time=6.05 ms
64 bytes from 113.171.10.155: icmp_seq=2 ttl=58 time=4.58 ms
64 bytes from 113.171.10.155: icmp_seq=3 ttl=58 time=3.22 ms

--- 113.171.10.155 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 3.220/4.618/6.052/1.156 ms
ameshkov commented 9 months ago

Right, the subdivision logic at this moment is enabled for 4 countries only: US, Russia, China and India.

Interesting data, thank you! We'll experiment with it a bit, I am not yet sure that it should be solved. We should first figure out why Akamai returns Singapore addresses for this subnet, it's unclear from the BGP information what's so different with this prefix compared to other prefixes announced by AS45899.

trmdi commented 9 months ago

We should first figure out why Akamai returns Singapore addresses for this subnet

Because that ECS belongs to HCM which is near Singapore? I guess a ECS in HN should be better. Since my subnet is in Hanoi (the north of Vietnam) R9iiRwD

ameshkov commented 9 months ago

@trmdi HCM?

As I see both subnets (yours and the one that AG DNS uses for ECS) belongs to the same AS45899.

ameshkov commented 9 months ago

Ah okay I got it, HCM = Ho Chi Minh.

Interesting idea indeed.

trmdi commented 9 months ago

Sorry, HCM means Ho Chi Minh city. (In the south of Vietnam)

ameshkov commented 9 months ago

Maybe you're right, we should check how Akamai nameserver responses change depending on what location the subnet belongs to.

Internal task ID: AGDNS-1881

ameshkov commented 9 months ago

Just got one more idea. Vietnam has quite a lot of different subdivisions in MaxMind (it uses ISO 3166-2). According to the ISO 3166-2 standard, Vietnam is divided into 58 provinces and 5 municipalities, but for network routing it is probably not that important and it'd be enough to do a "north-south" split.

@trmdi does that sound reasonable to you? If so, how would you split the subdivisions between north/south?

I think we could probably do the same for large countries as well, split subdivisions between west/east/north/south/central in order to improve cache efficiency.

trmdi commented 9 months ago

Yes, I think splitting between west/east/north/south/central is fine. Specifically, Vietnam area is S-shaped, narrow and stretched from the north to the south, so I think splitting to north/south (the upper 50% area is the north) is enough for network routing.

ameshkov commented 9 months ago

Okay it's not that easy.

Something's strange with Akamai. They indicate in responses that they don't rely on ECS:

dig -t a starling-oversea.byteoversea.com. @8.8.8.8 +subnet=14.226.99.0/24

...
CLIENT-SUBNET: 14.226.99.0/24/0
...

Now one more strange thing.

There's another domain that I tested with: store.akamai.steamstatic.com.

For some reason via Google DNS Akamai responds very accurately with Cyprus IP addresses.

dig -t a store.akamai.steamstatic.com. @8.8.8.8

...
;; ANSWER SECTION:
store.akamai.steamstatic.com. 20 IN A   195.14.151.138
store.akamai.steamstatic.com. 20 IN A   195.14.151.155
...

What's strange is that when I pass a custom subnet via dig to Google DNS, it continues to respond with accurate Cyprus IP addresses.

dig -t a store.akamai.steamstatic.com. @8.8.8.8 +subnet=14.226.99.0/24

...
;; ANSWER SECTION:
store.akamai.steamstatic.com. 20 IN A   195.14.151.155
store.akamai.steamstatic.com. 20 IN A   195.14.151.138
...

I honestly do not understand what's going on there and how on earth could Akamai learn about my location considering that I am being routed to Google DNS servers located in Switzerland (and Google DNS does not have any servers in Cyprus anyways).

Checking what Google DNS server is being used and what ECS it sends to authoritative nameservers:

dig -t txt o-o.myaddr.l.google.com. @8.8.8.8 +subnet=14.226.99.0/24

...
;; ANSWER SECTION:
o-o.myaddr.l.google.com. 60 IN  TXT "172.253.237.133"
o-o.myaddr.l.google.com. 60 IN  TXT "edns0-client-subnet 14.226.99.0/24"
...
ameshkov commented 9 months ago

Here's what I suspect:

  1. In the case of Akamai nameservers, Google DNS sends my real subnet in ECS even if I specify the subnet in the query.
  2. Akamai only accepts ECS from a select list of DNS servers. They accept it from Google DNS and OpenDNS, but not from us or Quad9.
vttale commented 9 months ago

Your suspicious are both correct. They will only respond to ECS from known resolvers with whom they have a contractual relationship, and relationship they have with Akamai does not permit the resolvers to forward ECS from clients.

ameshkov commented 9 months ago

@vttale thank you for confirming my suspicion!

On a side note, this looks like a weird business decision for Akamai. They should've been interested in speeding up connections for all Internet users, not for a part of them.

ameshkov commented 9 months ago

Found it: https://community.akamai.com/customers/s/article/What-is-the-ECS-onboarding-agreement-for-No-Client-Side-ECS

image

Well, our ECS implementation cannot fulfil what they're requiring. I'll try to contact them and maybe find some middle ground.

vttale commented 9 months ago

On a side note, this looks like a weird business decision for Akamai. They should've been interested in speeding up connections for all Internet users, not for a part of them.

From their point of view, their mapping is proprietary information based on extensive measurement of the Internet. They don't want that trove of data to be easily walked through iterative ECS queries.