search

Ranger802004 / asusmerlin

ASUS Merlin
GNU General Public License v3.0
49 stars 5 forks source link

Does wildcard work for CNAMES or not? #53

Open DiDHack opened 1 week ago

DiDHack commented 1 week ago

Hello! I have installed dig. I tried to add *.youtube.com to the domain list. Traceroute for www.youtube.com (CNAME youtube-ui.l.google.com) goes not through VPN gateway. When I added exactly www.youtube.com to the list it catches CNAME of www.youtube.com

So log dnsmasq shows cnames too. Why don't catch it from this log? Or can we not be sure that next line is that we try to find?

Nov  9 19:09:14 dnsmasq[25588]: query[A] www.youtube.com from 192.168.1.2
Nov  9 19:09:14 dnsmasq[25588]: forwarded www.youtube.com to 192.168.50.1
Nov  9 19:09:14 dnsmasq[25588]: query[A] www.youtube.com from 192.168.1.2
Nov  9 19:09:14 dnsmasq[25588]: forwarded www.youtube.com to 8.8.8.8
Nov  9 19:09:14 dnsmasq[25588]: reply www.youtube.com is <CNAME>
Nov  9 19:09:14 dnsmasq[25588]: reply youtube-ui.l.google.com is <CNAME>
Nov  9 19:09:14 dnsmasq[25588]: reply wide-youtube.l.google.com is 173.194.220.198
Nov  9 19:09:14 dnsmasq[25588]: query[A] wide-youtube.l.google.com from 192.168.1.2
Nov  9 19:09:14 dnsmasq[25588]: cached wide-youtube.l.google.com is 173.194.220.198
Nov  9 19:09:14 dnsmasq[25588]: query[AAAA] wide-youtube.l.google.com from 192.168.1.2
Nov  9 19:09:14 dnsmasq[25588]: forwarded wide-youtube.l.google.com to 192.168.50.1
Nov  9 19:09:14 dnsmasq[25588]: query[AAAA] wide-youtube.l.google.com from 192.168.1.2
Nov  9 19:09:14 dnsmasq[25588]: forwarded wide-youtube.l.google.com to 8.8.8.8
Nov  9 19:09:14 dnsmasq[25588]: reply wide-youtube.l.google.com is 2a00:1450:400c:c0b::c6
DiDHack commented 1 week ago

We can use dig if we see in the log

reply www.youtube.com is <CNAME>
Ranger802004 commented 1 week ago

Let me clarify a couple things here. Dig will match CNAME records for full domains, not wildcards and only from lookups and not dnsmasq log. Wildcards are only collected from dnsmasq or AdGuardHome logging.

DiDHack commented 1 week ago

Yes. But if I add *.youtube.com to the domain list. We can check the dnsmasq log for reply www.youtube.com is <CNAME> lines and then add this to the list automatically (if cnames is enabled) for the next cron job or we can check it with nslookup and dig immediately and then add new IPs to the ip list.

Ranger802004 commented 1 week ago

I will look into it and see if it is technically feasible.

Gundabadec commented 1 week ago

@Ranger802004 Script automatically added new ou domain or append already added domain to random policy. 3.0.3v

14:21:24 domain_vpn_routing: Debug - CNAME: ou is already added to trackers
14:21:24 domain_vpn_routing: Debug - CNAME: ou is already added to trackers
14:21:24 domain_vpn_routing: Debug - CNAME: ou is already added to trackers

16:58:16 domain_vpn_routing: Query Policy - Adding CNAME: ou to jetbrains
16:58:16 domain_vpn_routing: Query Policy - Added CNAME: ou to jetbrains
16:58:16 domain_vpn_routing: Query Policy - Adding CNAME: ou to jetbrains
16:58:16 domain_vpn_routing: Query Policy - Added CNAME: ou to jetbrains
16:58:16 domain_vpn_routing: Debug - CNAME: ou is already added to jetbrains

before query

*.youtube.com
*.ggpht.com

after query

*.youtube.com
*.ggpht.comou
ou

And I don't understand how it resolve to google dns ip add DVR-youtube-v4 8.8.8.8 comment "ou,youtube.com"

Ranger802004 commented 1 week ago

Go ahead and remove it from the policy and if it comes back let me know. There were some issues surrounding this problem in a previous version that were resolved in v3.0.1.

Ranger802004 commented 1 week ago

I am doing some additional testing and I see where there can still be some erroneous data entering the policy, let me work to resolve that and release a fix.

Gundabadec commented 1 week ago

@Ranger802004 "I remove it every time, but it returns" Sometimes add DVR-youtube-v4 8.8.8.8 comment "ou,youtube.com" in ipset file returns without ou line in _domainlist

Almost in every policy add DVR-twitter-v4 8.8.8.8 comment "ou,twitter.com,x.com"

Ranger802004 commented 1 week ago

Do you have ou,youtube.com as a domain added in a policy?

Gundabadec commented 1 week ago

@Ranger802004 No. Just youtube.com now. But sometimes ou domain is added to policy domain list automatically, perhaps when running the query policy command. I have removed it manually, but the line like this: add DVR-policyname-v4 8.8.8.8 comment “ou, real domain, real domain 2” still appears in a random policy IP set.

Ranger802004 commented 1 week ago

Did you remove it using the commands or manually deleting it from the domainlist file? If you are manually deleting it from the domainlist file readd it back and use the UI to remove it from the policy.

Ranger802004 commented 1 week ago

v3.0.4-beta1 is available on the beta channel to resolve this issue.

Release Notes: v3.0.4-beta1 - 11/13/2024 Fixes:

Gundabadec commented 1 week ago

v.3.0.4-beta1

Before:

*.twimg.comou
*.twitter.com
*.x.com
ou
twitter.com
x.com

If I try to remove ou from cli

After:

*.twitter.com
*.x.com
twitter.com
x.com

don't know why *.twimg.comou removed too But ou still appears randomly.

Second example: before:

*.ggpht.com
*.googlevideo.com
*.youtube.com
*.ytimg.com
ou
youtube.com

after remove ou domain:

*.ggpht.com
*.googlevideo.com
*.ytimg.com

maybe dig log helps with 8.8.8.8 in ipset

dig youtube.com
;; communications error to 8.8.8.8#53: timed out
;; communications error to 8.8.8.8#53: timed out
;; communications error to 8.8.8.8#53: timed out

; <<>> DiG 9.18.24 <<>> youtube.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 61101
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;youtube.com.           IN  A

;; ANSWER SECTION:
youtube.com.        300 IN  A   216.58.212.142

;; Query time: 86 msec
;; SERVER: 8.8.4.4#53(8.8.4.4) (UDP)
;; WHEN: Thu Nov 14 10:58:19 MSK 2024
;; MSG SIZE  rcvd: 56
Ranger802004 commented 1 week ago

3.0.4 uses the +short flag with dig with it should only output the replies only.

Ranger802004 commented 2 days ago

After some additional testing on this issue I found some more issues that were causing erroneous entries from CNAME coming in from the dig output. I have published v3.0.4-beta2 to the beta update channel to address these. Please test with this version and report any issues you experience.

Release Notes: v3.0.4-beta2 - 11/19/2024 Fixes: