search

ko-zu / psl

publicsuffixlist for python
Mozilla Public License 2.0
61 stars 10 forks source link

publicsuffix of cloudfront.net #17

Closed 4ft35t closed 4 years ago

4ft35t commented 4 years ago

cloudfront.net is a public suffix and belong to Amazon. but before the TLD was registered, Amazon also has the domain cloudfront with TLD .net. So it's confused to discern the root domain of *.cloudfront.net.

examples:

In [164]: ps.privatesuffix('d2os3n5ieuk9g5.cloudfront.net')
Out[164]: 'd2os3n5ieuk9g5.cloudfront.net'

In [165]: ps.privatesuffix('a286330aad4e096be6cdda229527774f4.profile.tlv50-c1.cloudfront.net')
Out[165]: 'tlv50-c1.cloudfront.net'

And we known every root domain has NS record, so check it.

dig d2os3n5ieuk9g5.cloudfront.net NS

; <<>> DiG 9.10.6 <<>> d2os3n5ieuk9g5.cloudfront.net NS
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1735
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 3

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4000
;; QUESTION SECTION:
;d2os3n5ieuk9g5.cloudfront.net. IN  NS

;; ANSWER SECTION:
d2os3n5ieuk9g5.cloudfront.net. 830 IN   NS  ns-1961.awsdns-53.co.uk.
d2os3n5ieuk9g5.cloudfront.net. 830 IN   NS  ns-1525.awsdns-62.org.
d2os3n5ieuk9g5.cloudfront.net. 830 IN   NS  ns-765.awsdns-31.net.
d2os3n5ieuk9g5.cloudfront.net. 830 IN   NS  ns-224.awsdns-28.com.

;; ADDITIONAL SECTION:
ns-1961.awsdns-53.co.uk. 2488   IN  A   205.251.199.169
ns-1525.awsdns-62.org.  8341    IN  A   205.251.197.245

;; Query time: 36 msec
;; SERVER: 10.95.44.53#53(10.95.44.53)
;; WHEN: Wed Sep 09 13:12:46 CST 2020
;; MSG SIZE  rcvd: 227
dig tlv50-c1.cloudfront.net NS

; <<>> DiG 9.10.6 <<>> tlv50-c1.cloudfront.net NS
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 868
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4000
;; QUESTION SECTION:
;tlv50-c1.cloudfront.net.   IN  NS

;; AUTHORITY SECTION:
cloudfront.net.     59  IN  SOA ns-418.awsdns-52.com. hostmaster.cloudfront.net. 1377556270 16384 2048 1048576 60

;; Query time: 1018 msec
;; SERVER: 10.95.44.53#53(10.95.44.53)
;; WHEN: Wed Sep 09 13:13:56 CST 2020
;; MSG SIZE  rcvd: 119
nslookup a286330aad4e096be6cdda229527774f4.profile.tlv50-c1.cloudfront.net 8.8.8.8
Server:     8.8.8.8
Address:    8.8.8.8#53

Non-authoritative answer:
Name:   a286330aad4e096be6cdda229527774f4.profile.tlv50-c1.cloudfront.net
Address: 13.226.6.197
Name:   a286330aad4e096be6cdda229527774f4.profile.tlv50-c1.cloudfront.net
Address: 13.226.6.231
Name:   a286330aad4e096be6cdda229527774f4.profile.tlv50-c1.cloudfront.net
Address: 13.226.6.22
Name:   a286330aad4e096be6cdda229527774f4.profile.tlv50-c1.cloudfront.net
Address: 13.226.6.45

tlv50-c1.cloudfront.net has no NS recored, but a286330aad4e096be6cdda229527774f4.profile.tlv50-c1.cloudfront.net has A recored, so the root domain of a286330aad4e096be6cdda229527774f4.profile.tlv50-c1.cloudfront.net is cloudfront.net.

ghost commented 4 years ago

If you want to trigger amendments/corrections in the PSL itself, refer to https://github.com/publicsuffix/list / https://publicsuffix.org/submit/

ko-zu commented 4 years ago

This repo contains just a copy of official PSL from https://github.com/publicsuffix/list .

And I can confirm this is expected behavior since cloudfront.net was submitted by Amazon as a private Public Suffix (=privately owned but publicly available for users). You can try only_icann=True flag to ignore this sort of Public Suffixes. https://github.com/ko-zu/psl#icann-and-private-suffixes