AdguardTeam / AdGuardHome

Network-wide ads & trackers blocking DNS server
https://adguard.com/adguard-home.html
GNU General Public License v3.0
23.81k stars 1.75k forks source link

DNS rewrite doesn't work for internal CNAME values #4057

Closed jamesmacwhite closed 2 years ago

jamesmacwhite commented 2 years ago

Issue Details

AdGuardHome -v --version
AdGuard Home
Version: v0.107.0
Channel: release
Go version: go1.17.3
Build time: 2021-12-28T18:09:17Z+0000
GOOS: linux
GOARCH: arm
Race: false
Dependencies:
        github.com/AdguardTeam/dnsproxy@v0.39.13 (sum: h1:7YM5Mr4EpFZ8UO4/4xd6zBG3lZ6AzZO6Xq29Cr4ydOY=)
        github.com/AdguardTeam/golibs@v0.10.3 (sum: h1:FBgk17zf35ESVWQKIqEUiqqB2bDaCBC8X5vMU760yB4=)
        github.com/AdguardTeam/urlfilter@v0.15.1 (sum: h1:dP6S7J6eFAk8MN4IDpUq2fZoBo8K8fmc6pXpxNIv84M=)
        github.com/NYTimes/gziphandler@v1.1.1 (sum: h1:ZUDjpQae29j0ryrS0u/B8HZfJBtBQHjqw2rQ2cqUQ3I=)
        github.com/aead/chacha20@v0.0.0-20180709150244-8b13a72661da (sum: h1:KjTM2ks9d14ZYCvmHS9iAKVt9AyzRSqNU1qabPih5BY=)
        github.com/aead/poly1305@v0.0.0-20180717145839-3fee0db0b635 (sum: h1:52m0LGchQBBVqJRyYYufQuIbVqRawmubW3OFGqK1ekw=)
        github.com/ameshkov/dnscrypt/v2@v2.2.2 (sum: h1:lxtS1iSA2EjTOMToSi+2+rwspNA+b/wG5/JpccvE9CU=)
        github.com/ameshkov/dnsstamps@v1.0.3 (sum: h1:Srzik+J9mivH1alRACTbys2xOxs0lRH9qnTA7Y1OYVo=)
        github.com/beefsack/go-rate@v0.0.0-20200827232406-6cde80facd47 (sum: h1:M57m0xQqZIhx7CEJgeLSvRFKEK1RjzRuIXiA3HfYU7g=)
        github.com/cheekybits/genny@v1.0.0 (sum: h1:uGGa4nei+j20rOSeDeP5Of12XVm7TGUd4dJA9RDitfE=)
        github.com/digineo/go-ipset/v2@v2.2.1 (sum: h1:k6skY+0fMqeUjjeWO/m5OuWPSZUAn7AucHMnQ1MX77g=)
        github.com/fsnotify/fsnotify@v1.4.9 (sum: h1:hsms1Qyu0jgnwNXIxa+/V/PDsU6CfLf6CNO8H7IWoS4=)
        github.com/go-ping/ping@v0.0.0-20210506233800-ff8be3320020 (sum: h1:mdi6AbCEoKCA1xKCmp7UtRB5fvGFlP92PvlhxgdvXEw=)
        github.com/google/go-cmp@v0.5.5 (sum: h1:Khx7svrCpmxxtHBq5j2mp/xVjsi8hQMfNLvJFAlrGgU=)
        github.com/google/gopacket@v1.1.19 (sum: h1:ves8RnFZPGiFnTS0uPQStjwru6uO6h+nlr9j6fL7kF8=)
        github.com/google/renameio@v1.0.1 (sum: h1:Lh/jXZmvZxb0BBeSY5VKEfidcbcbenKjZFzM/q0fSeU=)
        github.com/AdguardTeam/dhcp@v0.0.0-20210519141215-51808c73c0bf (sum: h1:gc042VRSIRSUzZ+Px6xQCRWNJZTaPkomisDfUZmoFNk=)
        github.com/joomcode/errorx@v1.0.3 (sum: h1:3e1mi0u7/HTPNdg6d6DYyKGBhA5l9XpsfuVE29NxnWw=)
        github.com/josharian/native@v0.0.0-20200817173448-b6b71def0850 (sum: h1:uhL5Gw7BINiiPAo24A2sxkcDI0Jt/sqp1v5xQCniEFA=)
        github.com/kardianos/service@v1.2.0 (sum: h1:bGuZ/epo3vrt8IPC7mnKQolqFeYJb7Cs8Rk4PSOBB/g=)
        github.com/lucas-clemente/quic-go@v0.21.1 (sum: h1:uuhCcu885TE9u/piPYMChI/yqA1lXfaLUEx8uCMxf8w=)
        github.com/marten-seemann/qtls-go1-17@v0.1.0-beta.1.2 (sum: h1:SficYjyOthSrliKI+EaFuXS6HqSsX3dkY9AqxAAjBjw=)
        github.com/mdlayher/ethernet@v0.0.0-20190606142754-0394541c37b7 (sum: h1:lez6TS6aAau+8wXUP3G9I3TGlmPFEq2CTxBaRqY6AGE=)
        github.com/mdlayher/netlink@v1.4.0 (sum: h1:n3ARR+Fm0dDv37dj5wSWZXDKcy+U0zwcXS3zKMnSiT0=)
        github.com/mdlayher/raw@v0.0.0-20210412142147-51b895745faf (sum: h1:InctQoB89TIkmgIFQeIL4KXNvWc1iebQXdZggqPSwL8=)
        github.com/ameshkov/dns@v1.1.32-0.20211214123418-7a5e0dc5f1b0 (sum: h1:a6ca3WlDG4zvUWqVFpVu48b9NZJ0fUFlRhiZKKkq+aw=)
        github.com/patrickmn/go-cache@v2.1.0+incompatible (sum: h1:HRMgzkcYKYpi3C8ajMPV8OFXaaRUnok+kx1WdO15EQc=)
        github.com/pkg/errors@v0.9.1 (sum: h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=)
        github.com/satori/go.uuid@v1.2.0 (sum: h1:0uYX9dsZ2yD7q2RtLRtPSdGDWzjeM3TbMJP9utgA0ww=)
        github.com/ti-mo/netfilter@v0.4.0 (sum: h1:rTN1nBYULDmMfDeBHZpKuNKX/bWEXQUhe02a/10orzg=)
        github.com/u-root/u-root@v7.0.0+incompatible (sum: h1:u+KSS04pSxJGI5E7WE4Bs9+Zd75QjFv+REkjy/aoAc8=)
        go.etcd.io/bbolt@v1.3.6 (sum: h1:/ecaJf0sk1l4l6V4awd65v2C3ILy7MSj+s/x1ADCIMU=)
        golang.org/x/crypto@v0.0.0-20210817164053-32db794688a5 (sum: h1:HWj/xjIHfjYU5nVXpTM0s39J9CbLn7Cc5a7IC5rwsMQ=)
        golang.org/x/net@v0.0.0-20210929193557-e81a3d93ecf6 (sum: h1:Z04ewVs7JhXaYkmDhBERPi41gnltfQpMWDnTnQbaCqk=)
        golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c (sum: h1:5KslGYwFpkhGh+Q16bwMP3cOontH8FOep7tGV86Y7SQ=)
        golang.org/x/sys@v0.0.0-20210909193231-528a39cd75f3 (sum: h1:3Ad41xy2WCESpufXwgs7NpDSu+vjxqLt2UFqUV+20bI=)
        golang.org/x/text@v0.3.7 (sum: h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=)
        gopkg.in/natefinch/lumberjack.v2@v2.0.0 (sum: h1:1Lc07Kr7qY4U2YPouBjpCLxpiyxIVoxqXgkXLknAOE8=)
        gopkg.in/yaml.v2@v2.4.0 (sum: h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=)
        howett.net/plist@v0.0.0-20201203080718-1454fab16a06 (sum: h1:QDxUo/w2COstK1wIBYpzQlHX/NqaQTcf9jyz347nI58=)

Expected Behaviour

Does the DNS rewrite function support rewriting an external domain request to an internal CNAME value?

E.g. subdomain.example.com -> subdomain.internal.example.com

The reason for this is several subdomains can be told to use the local IP address on the LAN rather than the external IPv4/IPv6 records i.e. bypassing reverse proxy. At the moment I have to set the same local A and AAAA value multiple times, it would be easier to be able to set an internal CNAME value, however it appears this doesn't seem to work.

Actual Behaviour

When trying to use an internal CNAME value, the DNS response is empty, it appears that the DNS rewrite is querying the external nameservers for the internal CNAME provided. When setting a specific local A or AAAA value, the DNS rewrite works.

TechieDylan commented 2 years ago

I'd recommend trying the latest stable release v0.107.2 as v0.107.1 had changes related to rewrites.

https://github.com/AdguardTeam/AdGuardHome/releases/tag/v0.107.1

jamesmacwhite commented 2 years ago

@TechieDylan Thank you for highlighting this, two further stable releases already!

I can see there is a specific note around CNAMEs and DNS Rewrite so I'll give it shot, thank you!

jamesmacwhite commented 2 years ago

It seems the issue is still there for me with DNS rewrites on 0.107.2.

It's basically when using an internal CNAME for a rewrite it seems to query the external nameservers which returns nothing. setting a local IP as a A or AAAA records work fine.

Lovely-Maisonette commented 2 years ago

I can confirm this stopped working with the 0.107 versions, including 0.107.2 (and also with the 108b, with one commit mentioning a rollback to fix the regression, but not fixing anything) Interestingly it does work in the TEST Rule page!

image

but not on the DNS queries, which now instead of rewriting, forwards the requests to my router.

image image

SlothCroissant commented 2 years ago

Can also confirm - CNAMEing example.com > host.lan causes the query to become "example.com.lan" and get piped out to the primary resolvers, which fails.

Any news on this front? Falling back to A records for now, but this breaks most internal CNAME solutions (which is what I imagine most using AdGuardHome are using for CNAMEs).

Happy to get logs, etc if needed, but the above seems to show it quite clearly.

EDIT: Here's a repro on v0.107.2:

Filtering rules:

||example.com^$dnsrewrite=NOERROR;CNAME;something.lan
||something.lan^$dnsrewrite=NOERROR;A;1.2.3.4

Result:

user@laptop ~ % dig example.com

; <<>> DiG 9.10.6 <<>> example.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 54209
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;example.com.           IN  A

;; AUTHORITY SECTION:
.           86400   IN  SOA a.root-servers.net. nstld.verisign-grs.com. 2022011802 1800 900 604800 86400

;; Query time: 30 msec
;; SERVER: 10.1.10.2#53(10.1.10.2)
;; WHEN: Tue Jan 18 22:30:48 CST 2022
;; MSG SIZE  rcvd: 112

The rules processor on the UI shows it's good:

image

However AdGuardHome shows what's really happening:

Screen Shot 2022-01-18 at 22 32 16

Also, not sure it matters, but something.lan resolves fine:

user@laptop ~ % dig something.lan

; <<>> DiG 9.10.6 <<>> something.lan
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10160
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;something.lan.         IN  A

;; ANSWER SECTION:
something.lan.      10  IN  A   1.2.3.4

;; Query time: 4 msec
;; SERVER: 10.1.10.2#53(10.1.10.2)
;; WHEN: Tue Jan 18 22:34:13 CST 2022
;; MSG SIZE  rcvd: 47
ainar-g commented 2 years ago

Apologies for the slow response.

@jamesmacwhite, could you please elaborate on what you mean by “internal” hosts? I assume that those are hosts that cannot be resolved by the external DNS servers?

If so, have you tried adding an upstream for it? For example:

[/internalnetwork.local/]192.168.1.53

Where internalnetwork.local is the domain name for such hosts and 192.168.1.53 is the DNS server that can resolve them.

jamesmacwhite commented 2 years ago

@ainar-g Yes. To clarify, I'm referring to specifying an internal CNAME domain.

Let's say you have app.example.com but you want requests on the LAN to the external domain to use an internal CNAME instead so app.internal.domain.com.

Currently it looks like when specifying an internal CNAME it ends up going to the external DNS resolvers and fails because it doesn't exist externally.

I do already have the appropriate nameservers set and "Use private reverse DNS resolvers" enabled so AdGuard Home should be able to resolve the internal CNAME. It looks like it just queries the external DNS resolvers regardless.

ainar-g commented 2 years ago

@jamesmacwhite

I do already have the appropriate nameservers set and "Use private reverse DNS resolvers" enabled so AdGuard Home should be able to resolve the internal CNAME. It looks like it just queries the external DNS resolvers regardless.

The upstream for your internal domain must be in the main “Upstream DNS servers” input, not in the “Private reverse DNS servers” one. As the name suggests, the latter is only for the reverse DNS queries. I.e. for queries like nslookup 192.168.1.2, not nslookup -type=cname my.internal.domain.

jamesmacwhite commented 2 years ago

Sorry for confusing reverse DNS, I have added my internal domain to Upstream Servers, but it's the same behaviour I'm afraid. Others have posted what looks like the DNS query hitting the external domain resolvers not being resolved internally. I don't know if that's what is happening, but based on dig DNS test the NS data is responding with the external DNS side not internal, so that might be where it's going wrong.

Edit: Ignore this, I think it was DNS caching related, it does look like it's working when the internal domain is added to Upstream Server. Whoops, my mistake, thanks for pointing that out.

ainar-g commented 2 years ago

Just to be clear, is the problem resolved?

SlothCroissant commented 2 years ago

@ainar-g What if we don't want to use an upstream resolver for internal names, but use the rewrites from AdGuardHome instead? In my case, I have automation that creates A records for all my internal hosts (my DHCP servers are on the same host as my AdGuardHome instances) - I don't have an upstream router that can resolve the internal names.

So the flow should be client asks for example.com > AdGuardHome responds with CNAME to someotherurl.lan (due to the DNS rewrite) > AdGuardHome replies with the A record that matches someotherurl.lan

This is still not functioning for me.

ainar-g commented 2 years ago

@SlothCroissant, please open a new discussion and elaborate on your use case.

SlothCroissant commented 2 years ago

@ainar-g my issue matches this one, my use case may just be different. Maybe I explained it poorly - let me try again and if I need a new issue, do let me know and I'll be happy to spin one up:

Issue: when querying a FQDN that will result in an internal CNAME response, the resulting request (which should be an A record) is never replied, it just sends the resulting A record request to the default upstream.

Example (this can be repro'd by anyone, these are the actual rules I'm using to show the behavior):

I have the following DNS rules:

||example.com^$dnsrewrite=NOERROR;CNAME;something.lan
||something.lan^$dnsrewrite=NOERROR;A;1.2.3.4

So if I query example.com, I'd expect my client to end up receiving a CNAME for something.lan, and a subsequent A record for 1.2.3.4. Logs to show the behavior:

I queried example.com from a client, and AGH kicked it upstream and returned "non-existent domain":

{
    "T": "2022-01-24T10:52:06.985941257-06:00",
    "QH": "example.com",
    "QT": "A",
    "QC": "IN",
    "CP": "",
    "Answer": "9kiFgwABAAAAAQABB2V4YW1wbGUDY29tAAABAAEAAAYAAQABUYAAPQFhDHJvb3Qtc2VydmVycwNuZXQABW5zdGxkDHZlcmlzaWduLWdyc8AUeIV18AAABwgAAAOEAAk6gAABUYAAACkQAAAAAAAAAA==",
    "Result": {
        "Reason": 11,
        "Rules": [{
                "Text": "||example.com^$dnsrewrite=NOERROR;CNAME;something.lan"
            }
        ],
        "CanonName": "something.lan"
    },
    "Upstream": "https://1.0.0.1:443/dns-query",
    "IP": "10.1.10.9",
    "Elapsed": 27255229
}

Also as a datapoint - something.lan resolves just fine if you query it directly:

{
    "T": "2022-01-24T11:01:05.016137113-06:00",
    "QH": "something.lan",
    "QT": "AAAA",
    "QC": "IN",
    "CP": "",
    "Answer": "AAOBgAABAAAAAAAACXNvbWV0aGluZwNsYW4AABwAAQ==",
    "Result": {
        "Reason": 11,
        "Rules": [{
                "Text": "||something.lan^$dnsrewrite=NOERROR;A;1.2.3.4"
            }
        ],
        "DNSRewriteResult": {
            "Response": {
                "1": ["1.2.3.4"]
            }
        }
    },
    "IP": "10.1.10.101",
    "Elapsed": 27778
}

So this feels like the same issue - a CNAME to an internal DNS record never returns properly, and instead is sent upstream (where it fails, as .lan isn't a valid TLD).

Thoughts? I can create a fresh issue if you need.

jamesmacwhite commented 2 years ago

@ainar-g To reply to your comment, yes, it does work after applying my internal domain in Upstream servers, then internal CNAMEs with DNS rewrites are resolving for me.

ainar-g commented 2 years ago

@SlothCroissant, the solution that worked for OP should work for you too. Add something like:

[/lan/]YOUR_AGH_ADDR

to the main upstream server list. That will force AGH to look up the resulting CNAME in its own filtering lists.

I'll close this issue now. If that won't help, please open a new discussion.

SlothCroissant commented 2 years ago

Worked like a charm. Wasn't aware that you could set AGH to query itself. Thanks!

ainar-g commented 2 years ago

As long as you avoid infinite recursion, heh. You're welcome.

TheModin commented 2 years ago

I can not get this to work. So I add something like: [/domainxyz.com/]192.168.1.110 To the upstream server list. And have a DNS rewrite for: test.domainxyz.com I get the following error. image I'm clearly doing something wrong since the rest of you got it working.

SlothCroissant commented 2 years ago

What is the query type on that 10sec timout query? Mine were all AAAA, and I don't use IPv6 yet in my network. So I had to add this to my AGH rules:

||lan^$dnstype=AAAA

So that the follow-up AAAA query didn't get forwarded along and failed.

TheModin commented 2 years ago

@SlothCroissant thanks. I get that on both A and AAAA. I tried adding: ||domainxyz.com^$dnstype=AAAA But still getting the same error message.

pyvarex commented 2 years ago

Can anyone post his working solution? Can't seem to get te right rules in place. Just like some more ppl :)

Running latest edge version

SlothCroissant commented 2 years ago

I can post my config details - traveling at the moment but can try to find some time to grab the details and post here this week sometime via mobile.

tesla877 commented 2 years ago

Running into same issue, can you post your config @SlothCroissant ? Thx

tesla877 commented 2 years ago

Okay - I got this to work by setting both domains to resolve locally. I think the resolver was trying to externally resolve the rewritten domain (makes sense because I didn't ask AG to resolve it locally).

DNS Settings: [/dev.test/]127.0.0.1 [/money.talks/]127.0.0.1

DNS Rewrite: dev.test > money.talks

If you only want CNAME or A responses, add this to your filtering rules: ||dev.test^$dnstype=~A|~CNAME

SlothCroissant commented 2 years ago

Sorry for the delay - here's what I have (this is actually deployed, not a "find and replace for obfuscation" thing):

(this is in my adguardhome.yaml file, btw):

user_rules:
- '||example.com^$dnsrewrite=NOERROR;CNAME;something.lan'
- '||something.lan^$dnsrewrite=NOERROR;A;1.2.3.4'
- '||lan^$dnstype=AAAA'
upstream_dns:
  - https://1.1.1.1/dns-query
  - https://1.0.0.1/dns-query
  - '[/lan/]127.0.0.1'

This tells agh to return a CNAME of something.lan when I query for example.com. Then to return 1.2.3.4 when trying to resolve something.lan. It also tells agh to not ever reply with AAAA records for any .lan hosts, and to use itself as the upstream resolver for anything ending in .lan.

Let me know if this helps. The end behavior is that I don't get all the SERVFAILs any longer.

Here's the result in dig:

ryanb@docker:~$ dig example.com

; <<>> DiG 9.16.22-Debian <<>> example.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4380
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;example.com.                   IN      A

;; ANSWER SECTION:
example.com.            3600    IN      CNAME   something.lan.
something.lan.          3269    IN      A       1.2.3.4

;; Query time: 0 msec
;; SERVER: 10.1.10.2#53(10.1.10.2)
;; WHEN: Thu Mar 10 10:34:41 CST 2022
;; MSG SIZE  rcvd: 83
; <<>> DiG 9.16.22-Debian <<>> example.com -q aaaa
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 14179
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;example.com.                   IN      A

;; ANSWER SECTION:
example.com.            3600    IN      CNAME   something.lan.
something.lan.          3401    IN      A       1.2.3.4

;; Query time: 0 msec
;; SERVER: 10.1.10.2#53(10.1.10.2)
;; WHEN: Thu Mar 10 10:32:29 CST 2022
;; MSG SIZE  rcvd: 83

;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 43325
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;aaaa.                          IN      A

;; AUTHORITY SECTION:
.                       86395   IN      SOA     a.root-servers.net. nstld.verisign-grs.com. 2022031000 1800 900 604800 86400

;; Query time: 4 msec
;; SERVER: 10.1.10.2#53(10.1.10.2)
;; WHEN: Thu Mar 10 10:32:29 CST 2022
;; MSG SIZE  rcvd: 108