Closed polarathene closed 4 weeks ago
I now consider this resolved given the findings I've documented here below.
TL:DR (Update):
1.1.1.1
the queries were all being sent to router DNS (192.168.1.1
below) indirectly for WSL2 and Docker.TXT github.com
is too big for standard UDP query (512 byte size limit), while 1.1.1.1
does support TCP, and a truncated response informs the DNS client to request again over TCP where it succeeds.I am not sure how to reproduce the original error I reported in the issue title, but it does appear like it was specific to my environment and router / DNS concerns discovered that I've noted in the summary.
Summary:
TXT github.com
query. The WSL2 / Docker DNS services did not bubble that up for visibility.0.0.0.0
configured as a 2ndry DNS service (even if there was no alternative DNS service available), where DNS queries could be observed being sent across both. When the 2nd DNS service was 1.1.1.1
both Windows and WSL2 output correct results for TXT github.com
, but the Docker container did not. Presumably the difference here is how they integrate/relay with the host DNS that they forward the queries to and receive responses from.Summary with a bit more context/insights:
At least with the attempted reproduction below, it was discovered that the Windows host had the router DNS as the primary DNS with a secondary DNS as 0.0.0.0
. Two insights were found related to this:
doggo --debug
on the Windows 11 host output, WSL2 and Docker were unaware), and the query appeared to be in parallel which explains why my original investigation into DNS issues were racey with inconsistent results.0.0.0.0
would resolve to 127.0.0.1
for DNS service. I had CoreDNS (started from WSL2 via Docker container) that had a fallback rule to forward unresolved queries to 1.1.1.1
. It's possible this container was also running in the background when I originally reported, but I was assuming without anything explicitly configured to use it at both times that it was not receiving queries.The published port 53:53
for the container was not accessible within WSL2 network interfaces.
127.0.0.53:53
? (Docker runs it's own DNS somewhere at 127.0.0.11:53
too IIRC, perhaps on the WSL2 docker image that Docker Desktop manages).0.0.0.0
(all addresses) with the docker port publishing syntax. If I instead explicitly configure as 127.0.0.1:53:53
then ss -tulpn
lists the binding within WSL2. Doing so makes the query fail on Windows 11 host (PowerShell), whilst I can bind to the private network IP assigned to that WSL2 distro instead, and explicitly query that for Windows 11 and Docker containers to reach.127.0.0.53
in WSL2 seems to forward to /etc/resolv.conf
when queried, thus behaves the same as when querying without an explicit name server.Error output for TXT github.com
query has fluctuated between these two (probably due parallel query on host?):
$ ./doggo TXT github.com @127.0.0.53
time=2024-08-07T14:36:03.986+12:00 level=ERROR msg="error in lookup" error="read udp 127.0.0.1:42247->127.0.0.53:53: i/o timeout"
$ ./doggo TXT github.com
time=2024-08-07T14:38:59.708+12:00 level=ERROR msg="Error looking up DNS records" error="context deadline exceeded"
A github.com
query works fine. Pretty sure the TXT
record failure is related to the large record that is technically multi-part and the router DNS server just doesn't handle that (it's a budget router known to have issues that require manual restarts).
TXT github.com
queries. WSL2 and the Docker container could all reach 192.168.1.1
to produce this error that only Windows 11 was originally outputting. Presumably from the indirection that error did not bubble back up through the WSL2 and Docker internal DNS services.For context my WSL2 install is not the newer 2.2.1 release (which has DNS improvements that'd likely change the findings reported here):
$ wsl --version
WSL version: 2.0.0.0
Kernel version: 5.15.123.1-1
WSLg version: 1.0.57
MSRDC version: 1.2.4485
Direct3D version: 1.608.2-61064218
DXCore version: 10.0.25880.1000-230602-1350.main
Windows version: 10.0.22631.3810
If you landed here due to dns: overflowing header size
, it seems this topic is discussed well at CoreDNS, the query like TXT github.com
over UDP is larger than 512 bytes (for github query is was over 1600), but that should implicitly query again over TCP as a workaround if the response had a truncate bit (tc):
https://github.com/coredns/coredns/pull/6003#discussion_r1154812126
// If the error is an overflow, we probably have a misbehaving upstream that is sending out-of-spec (>512 byte) UDP responses without an eDNS0 OPT RR. // Set the truncate bit and return the result
As an update to this, I tried again to reproduce out of curiosity. I found that the latest release of doggo
did not output the error, just blank results.
I wondered if this was a change related to doggo
so I double-checked with q
, which still has not published any newer releases, so for reference q
in both cases was definitely 0.19.2
(Jan 2024), and I used the equivalent GHCR image. This too produced empty output.
$ ./doggo TXT github.com
NAME TYPE CLASS TTL ADDRESS NAMESERVER
# Doesn't look like doggo is aware of the issue:
$ ./doggo --debug TXT github.com
time=2024-08-06T22:49:44.342Z level=DEBUG msg="LoadNameservers: Initial nameservers" nameservers=[]
time=2024-08-06T22:49:44.342Z level=DEBUG msg="No user specified nameservers, falling back to system nameservers"
time=2024-08-06T22:49:44.342Z level=DEBUG msg="Loaded system nameservers" nameservers="[{Address:192.168.65.7:53 Type:udp}]"
time=2024-08-06T22:49:44.342Z level=DEBUG msg="initiating UDP resolver"
time=2024-08-06T22:49:44.342Z level=DEBUG msg="Attempting to resolve" domain=github.com. ndots=0 nameserver=192.168.65.7:53
# Neither output anything:
$ docker run --rm -it ghcr.io/natesales/q TXT github.com --txtconcat
$ docker run --rm -it ghcr.io/natesales/q TXT github.com
So given that q
hasn't changed, perhaps something else has.
26.1.1
(April 2024 release).With doggo 1.0.4
:
So both Windows 11 host and WSL2 have no issue with the TXT github.com
query. The windows 11 host does emit a lookup error before outputting results however.
192.168.1.1
for that error is assigned as the DNS server to my laptop wifi interface that is handling the internet connection. So querying it explicitly:
# Need to quote for PowerShell, else syntax error due to shell feature:
$ ./doggo.exe TXT github.com '@192.168.1.1'
time=2024-08-07T11:20:42.897+12:00 level=ERROR msg="error in lookup" error="dial tcp 192.168.1.1:53: connectex: No connection could be made because the target machine actively refused it."
NAME TYPE CLASS TTL ADDRESS NAMESERVER
Likewise when done on WSL2:
$ ./doggo TXT github.com @192.168.1.1
time=2024-08-07T11:22:00.427+12:00 level=ERROR msg="error in lookup" error="dial tcp 192.168.1.1:53: connect: connection refused"
NAME TYPE CLASS TTL ADDRESS NAMESERVER
And for additional context, within a Docker container (volume mounting the same doggo 1.0.4 github release binary used with WSL2):
$ docker run --rm -v ./doggo:/opt/doggo:ro --workdir /opt alpine ash -c './doggo TXT github.com @192.168.1.1'
time=2024-08-06T23:23:01.328Z level=ERROR msg="error in lookup" error="dial tcp 192.168.1.1:53: connect: connection refused"
NAME TYPE CLASS TTL ADDRESS NAMESERVER
And just so it's perfectly clear with that obvious cause, this only affects the TXT
record query, the docker container (or rather 192.168.1.1
DNS server) supports querying other records without issue (no error emitted):
$ docker run --rm -v ./doggo:/opt/doggo:ro --workdir /opt alpine ash -c './doggo A github.com @192.168.1.1'
NAME TYPE CLASS TTL ADDRESS NAMESERVER
github.com. A IN 3s 4.237.22.38 192.168.1.1:53
So some difference between A
and TXT
query there under the hood seems relevant?
On the Windows 11 when checking for the DNS Server it was configured with via PowerShell, there is this snippet:
$ ipconfig /all
# I've omitted/trimmed fields, but note the 2nd entry for DNS servers:
Wireless LAN adapter Wi-Fi:
Default Gateway . . . . . . . . . : 192.168.1.1
DHCP Server . . . . . . . . . . . : 192.168.1.1
DNS Servers . . . . . . . . . . . : 192.168.1.1
0.0.0.0
Running doggo via PowerShell with debug this time, we get this information before the results are printed:
$ ./doggo.exe --debug TXT github.com
time=2024-08-07T11:28:32.961+12:00 level=DEBUG msg="LoadNameservers: Initial nameservers" nameservers=[]
time=2024-08-07T11:28:32.961+12:00 level=DEBUG msg="No user specified nameservers, falling back to system nameservers"
time=2024-08-07T11:28:32.967+12:00 level=DEBUG msg="Loaded system nameservers" nameservers="[{Address:192.168.1.1:53 Type:udp} {Address:0.0.0.0:53 Type:udp}]"
time=2024-08-07T11:28:32.968+12:00 level=DEBUG msg="initiating UDP resolver"
time=2024-08-07T11:28:32.968+12:00 level=DEBUG msg="initiating UDP resolver"
time=2024-08-07T11:28:32.968+12:00 level=DEBUG msg="Attempting to resolve" domain=github.com. ndots=0 nameserver=0.0.0.0:53
time=2024-08-07T11:28:32.968+12:00 level=DEBUG msg="Attempting to resolve" domain=github.com. ndots=0 nameserver=192.168.1.1:53
time=2024-08-07T11:28:32.977+12:00 level=DEBUG msg="Response truncated; retrying now" protocol=tcp
time=2024-08-07T11:28:32.977+12:00 level=DEBUG msg="Attempting to resolve" domain=github.com. ndots=0 nameserver=192.168.1.1:53
time=2024-08-07T11:28:33.009+12:00 level=DEBUG msg="Response truncated; retrying now" protocol=tcp
time=2024-08-07T11:28:33.009+12:00 level=DEBUG msg="Attempting to resolve" domain=github.com. ndots=0 nameserver=0.0.0.0:53
time=2024-08-07T11:28:35.036+12:00 level=ERROR msg="error in lookup" error="dial tcp 192.168.1.1:53: connectex: No connection could be made because the target machine actively refused it."
0.0.0.0
is valid for bind/listen address to cover all interfaces, but typically as a target for requests IIRC? The ipconfig
output showed no other configured DNS services either so I'm a bit curious what happened differently here. Seems from the above debug logs that both queries were sent in parallel (which explains why in May when I raised this bug report I had weird/inconsistent/racey results when resolving crates.io
records with q
unless I explicit set the DNS server to something like 1.1.1.1
)..
# No error this time:
$ ./doggo.exe TXT github.com '@0.0.0.0'
NAME TYPE CLASS TTL ADDRESS NAMESERVER
github.com. TXT IN 578s "MS=6BF03E6AF5CB689E315FB6199603BABF2C88D805" 0.0.0.0:53
github.com. TXT IN 578s "MS=ms44452932" 0.0.0.0:53
github.com. TXT IN 578s "MS=ms58704441" 0.0.0.0:53
github.com. TXT IN 578s "adobe-idp-site-verification=b92c9e999aef825edc36e0a3d847d2dbad5b2fc0e05c79ddd7a16139b48ecf4b" 0.0.0.0:53
github.com. TXT IN 578s "apple-domain-verification=RyQhdzTl6Z6x8ZP4" 0.0.0.0:53
github.com. TXT IN 578s "atlassian-domain-verification=jjgw98AKv2aeoYFxiL/VFaoyPkn3undEssTRuMg6C/3Fp/iqhkV4HVV7WjYlVeF8" 0.0.0.0:53
github.com. TXT IN 578s "beautifulai-site-verification=e478d764-9335-4af3-ac7a-2d5ab61b59aa" 0.0.0.0:53
github.com. TXT IN 578s "calendly-site-verification=at0DQARi7IZvJtXQAWhMqpmIzpvoBNF7aam5VKKxP" 0.0.0.0:53
github.com. TXT IN 578s "docusign=087098e3-3d46-47b7-9b4e-8a23028154cd" 0.0.0.0:53
github.com. TXT IN 578s "facebook-domain-verification=39xu4jzl7roi7x0n93ldkxjiaarx50" 0.0.0.0:53
github.com. TXT IN 578s "google-site-verification=82Le34Flgtd15ojYhHlGF_6g72muSjamlMVThBOJpks" 0.0.0.0:53
github.com. TXT IN 578s "google-site-verification=UTM-3akMgubp6tQtgEuAkYNYLyYAvpTnnSrDMWoDR3o" 0.0.0.0:53
github.com. TXT IN 578s "krisp-domain-verification=ZlyiK7XLhnaoUQb2hpak1PLY7dFkl1WE" 0.0.0.0:53
github.com. TXT IN 578s "loom-site-verification=f3787154f1154b7880e720a511ea664d" 0.0.0.0:53
github.com. TXT IN 578s "miro-verification=d2e174fdb00c71e0bcf58f8e58c3da2dd80dcfa9" 0.0.0.0:53
github.com. TXT IN 578s "stripe-verification=f88ef17321660a01bab1660454192e014defa29ba7b8de9633c69d6b4912217f" 0.0.0.0:53
github.com. TXT IN 578s "v=spf1 ip4:192.30.252.0/22 0.0.0.0:53
include:_netblocks.google.com
include:_netblocks2.google.com
include:_netblocks3.google.com
include:spf.protection.outlook.com
include:mail.zendesk.com
include:_spf.salesforce.com
include:servers.mcsv.net
ip4:166.78.69.169 ip4:1"
"66.78.69.170 ip4:166.78.71.131
ip4:167.89.101.2
ip4:167.89.101.192/28
ip4:192.254.112.60
ip4:192.254.112.98/31
ip4:192.254.113.10
ip4:192.254.113.101
ip4:192.254.114.176
ip4:62.253.227.114 ~all"
Not the easiest thing to search for, but thankfully trying the same in linux land we get some insight:
$ docker run --rm -v ./doggo:/opt/doggo:ro --workdir /opt alpine ash -c './doggo A github.com @0.0.0.0'
time=2024-08-06T23:43:54.217Z level=ERROR msg="error in lookup" error="read udp 127.0.0.1:54335->127.0.0.1:53: read: connection refused"
NAME TYPE CLASS TTL ADDRESS NAMESERVER
And sure enough, on Windows 11 with powershell if I adjust that to query @127.0.0.1
the query succeeds. For additional context, on WSL2 I get the same error as shown within the Docker command output.
Now I can't recall if the following would apply to my original report in May (although it's now clear how the DNS server IP was likely the same as the one on the host as shown, but I had changed the host to use 1.1.1.1
, the Docker Daemon had not been restarted and thus might have tried to still query DNS from the router IP, or the original error may have been related to Docker embedded DNS and potentially it's own fallback, which may have been fixed if I have since upgraded Docker 🤷♂️ )
This additional DNS service at 127.0.0.1
on the host that neither WSL2 or the Docker container could reach (since duh, 127.0.0.1
is the local loopback interface on each one), I did realize while typing this update out that I had CoreDNS container running in the background 😂 it was publishing 53:53
(TCP) and 53:53/udp
(UDP), but WSL2 wouldn't show the ports published via ss -tulpn
like I'd usually expect to see... because that would imply *:53
for all interfaces, but WSL2 already had :53
from systemd at 127.0.0.53%lo:53
(both UDP and TCP), so Docker was like "I can't bind generically (0.0.0.0:53
/ *:
53)", yet this was still published and accessible in a way that the Windows 11 host could reach it....? (if I published with an explicit IP to bind the port mapping from that belonged to WSL2 that'd work and show up in ss -tulpn
output, thus it was due to the port conflict)
So I bring down that CoreDNS container... voila failure:
$ ./doggo.exe TXT github.com '@127.0.0.1'
time=2024-08-07T11:46:54.035+12:00 level=ERROR msg="error in lookup" error="read udp 127.0.0.1:63598->127.0.0.1:53: i/o timeout"
time=2024-08-07T11:46:54.036+12:00 level=ERROR msg="Error looking up DNS records" error="context deadline exceeded"
ipconfig /all
still lists 0.0.0.0
, and if I don't provide an explicit DNS server we get the expected debug failure:
./doggo.exe --debug TXT github.com
time=2024-08-07T11:47:09.949+12:00 level=DEBUG msg="LoadNameservers: Initial nameservers" nameservers=[]
time=2024-08-07T11:47:09.950+12:00 level=DEBUG msg="No user specified nameservers, falling back to system nameservers"
time=2024-08-07T11:47:09.956+12:00 level=DEBUG msg="Loaded system nameservers" nameservers="[{Address:192.168.1.1:53 Type:udp} {Address:0.0.0.0:53 Type:udp}]"
time=2024-08-07T11:47:09.956+12:00 level=DEBUG msg="initiating UDP resolver"
time=2024-08-07T11:47:09.956+12:00 level=DEBUG msg="initiating UDP resolver"
time=2024-08-07T11:47:09.956+12:00 level=DEBUG msg="Attempting to resolve" domain=github.com. ndots=0 nameserver=0.0.0.0:53
time=2024-08-07T11:47:09.956+12:00 level=DEBUG msg="Attempting to resolve" domain=github.com. ndots=0 nameserver=192.168.1.1:53
time=2024-08-07T11:47:09.965+12:00 level=DEBUG msg="Response truncated; retrying now" protocol=tcp
time=2024-08-07T11:47:09.965+12:00 level=DEBUG msg="Attempting to resolve" domain=github.com. ndots=0 nameserver=192.168.1.1:53
time=2024-08-07T11:47:12.018+12:00 level=ERROR msg="error in lookup" error="dial tcp 192.168.1.1:53: connectex: No connection could be made because the target machine actively refused it."
time=2024-08-07T11:47:14.958+12:00 level=ERROR msg="Error looking up DNS records" error="context deadline exceeded"
Both were queried but neither successful 👍
It worked before with CoreDNS only because I had my local custom zones with 1.1.1.1
as a forward query for fallback.
doggo
vs dig
These appear to be on par now, while my original report noted a discrepancy.
I am aware of dig
not using glibc IIRC? (at least that it and other DNS tools typically were more "direct", unlike other software that hooked into glibc
which affected DNS resolution a bit differently, perhaps via getaddrinfo()
?). In the past, Go based software with static builds when run on Alpine / musl have needed a workaround to include /etc/nsswitch.conf
, otherwise the query would differ from glibc
based images/distro (Alpine has been known to have various other DNS quirks too), but that has since been resolved.
TXT
likewise fails with dig
(NOTE: 172.18.160.1
is the DNS server in /etc/resolv.conf
on WSL2, it'll match to 192.168.1.1
on the host):
$ dig TXT github.com
;; Truncated, retrying in TCP mode.
;; Connection to 172.18.160.1#53(172.18.160.1) for github.com failed: timed out.
;; no servers could be reached
Again A
record query was successful:
$ dig A github.com
; <<>> DiG 9.18.28-0ubuntu0.22.04.1-Ubuntu <<>> A github.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 37724
;; flags: qr rd ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available
;; QUESTION SECTION:
;github.com. IN A
;; ANSWER SECTION:
github.com. 0 IN A 4.237.22.38
;; Query time: 20 msec
;; SERVER: 172.18.160.1#53(172.18.160.1) (UDP)
;; WHEN: Wed Aug 07 12:21:07 NZST 2024
;; MSG SIZE rcvd: 54
Reverse DNS seems to work too:
$ ./doggo -x 4.237.22.38
NAME TYPE CLASS TTL ADDRESS NAMESERVER
38.22.237.4.in-addr.arpa. PTR IN 0s github.com. 172.18.160.1:53
$ dig -x 4.237.22.38
; <<>> DiG 9.18.28-0ubuntu0.22.04.1-Ubuntu <<>> -x 4.237.22.38
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 51957
;; flags: qr rd ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available
;; QUESTION SECTION:
;38.22.237.4.in-addr.arpa. IN PTR
;; ANSWER SECTION:
38.22.237.4.in-addr.arpa. 0 IN PTR github.com.
;; Query time: 0 msec
;; SERVER: 172.18.160.1#53(172.18.160.1) (UDP)
;; WHEN: Wed Aug 07 12:24:23 NZST 2024
;; MSG SIZE rcvd: 90
Although not with doggo
in the Docker container:
$ docker run --rm -v ./doggo:/opt/doggo:ro --workdir /opt alpine ash -c './doggo --debug -x 4.237.22.38'
time=2024-08-07T00:29:42.802Z level=DEBUG msg="LoadNameservers: Initial nameservers" nameservers=[]
time=2024-08-07T00:29:42.802Z level=DEBUG msg="No user specified nameservers, falling back to system nameservers"
time=2024-08-07T00:29:42.802Z level=DEBUG msg="Loaded system nameservers" nameservers="[{Address:192.168.65.7:53 Type:udp}]"
time=2024-08-07T00:29:42.802Z level=DEBUG msg="initiating UDP resolver"
time=2024-08-07T00:29:42.802Z level=DEBUG msg="Attempting to resolve" domain=38.22.237.4.in-addr.arpa. ndots=0 nameserver=192.168.65.7:53
NAME TYPE CLASS TTL ADDRESS NAMESERVER
$ docker run --rm -v ./doggo:/opt/doggo:ro --workdir /opt alpine ash -c 'apk add bind-tools && dig -x 4.237.22.38'
; <<>> DiG 9.18.24 <<>> -x 4.237.22.38
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 20531
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;38.22.237.4.in-addr.arpa. IN PTR
;; Query time: 1050 msec
;; SERVER: 192.168.65.7#53(192.168.65.7) (UDP)
;; WHEN: Wed Aug 07 00:27:48 UTC 2024
;; MSG SIZE rcvd: 42
Now with Cloudflare 1.1.1.1
as DNS:
$ docker run --rm -v ./doggo:/opt/doggo:ro --workdir /opt alpine ash -c './doggo --debug -x 4.237.22.38 @1.1.1.1'
time=2024-08-07T00:30:54.542Z level=DEBUG msg="LoadNameservers: Initial nameservers" nameservers=[1.1.1.1]
time=2024-08-07T00:30:54.542Z level=DEBUG msg="Added nameserver" nameserver="{Address:1.1.1.1:53 Type:udp}"
time=2024-08-07T00:30:54.542Z level=DEBUG msg="LoadNameservers: Final nameservers" nameservers="[{Address:1.1.1.1:53 Type:udp}]"
time=2024-08-07T00:30:54.542Z level=DEBUG msg="initiating UDP resolver"
time=2024-08-07T00:30:54.543Z level=DEBUG msg="Attempting to resolve" domain=38.22.237.4.in-addr.arpa. ndots=0 nameserver=1.1.1.1:53
NAME TYPE CLASS TTL ADDRESS NAMESERVER STATUS
22.237.4.in-addr.arpa. SOA IN 156s ns1-04.azure-dns.com. 1.1.1.1:53 NXDOMAIN
azuredns-hostmaster.microsoft.com.
1 3600 300 2419200 300
$ docker run --rm -v ./doggo:/opt/doggo:ro --workdir /opt alpine ash -c 'apk add bind-tools && dig -x 4.237.22.38 @1.1.1.1'
; <<>> DiG 9.18.24 <<>> -x 4.237.22.38 @1.1.1.1
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 47403
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;38.22.237.4.in-addr.arpa. IN PTR
;; AUTHORITY SECTION:
22.237.4.in-addr.arpa. 125 IN SOA ns1-04.azure-dns.com. azuredns-hostmaster.microsoft.com. 1 3600 300 2419200 300
;; Query time: 10 msec
;; SERVER: 1.1.1.1#53(1.1.1.1) (UDP)
;; WHEN: Wed Aug 07 00:31:26 UTC 2024
;; MSG SIZE rcvd: 139
That seems to suggest that dig
got a cached response on WSL2 for the reverse DNS mapping to Github since that was resolved earlier?
Sure enough when I queried back on WSL2 we got the response that we'd get from 1.1.1.1
:
$ ./doggo -x 4.237.22.38
NAME TYPE CLASS TTL ADDRESS NAMESERVER STATUS
22.237.4.in-addr.arpa. SOA IN 299s ns1-04.azure-dns.com. 172.18.160.1:53 NXDOMAIN
azuredns-hostmaster.microsoft.com.
1 3600 300 2419200 300
$ ./doggo -x 4.237.22.38 @1.1.1.1
NAME TYPE CLASS TTL ADDRESS NAMESERVER STATUS
22.237.4.in-addr.arpa. SOA IN 57s ns1-04.azure-dns.com. 1.1.1.1:53 NXDOMAIN
azuredns-hostmaster.microsoft.com.
1 3600 300 2419200 300
By performing a query for the A
record like had been done earlier in this report, that affects the local resolution:
$ ./doggo A github.com
NAME TYPE CLASS TTL ADDRESS NAMESERVER
github.com. A IN 0s 4.237.22.38 172.18.160.1:53
$ ./doggo -x 4.237.22.38
NAME TYPE CLASS TTL ADDRESS NAMESERVER
38.22.237.4.in-addr.arpa. PTR IN 0s github.com. 172.18.160.1:53
# But as expected 1.1.1.1 is unaffected:
$ ./doggo -x 4.237.22.38 @1.1.1.1
NAME TYPE CLASS TTL ADDRESS NAMESERVER STATUS
22.237.4.in-addr.arpa. SOA IN 280s ns1-04.azure-dns.com. 1.1.1.1:53 NXDOMAIN
azuredns-hostmaster.microsoft.com.
1 3600 300 2419200 300
That doesn't change in the container with Dockers embedded DNS server (192.168.65.7
), thus the WSL2 (172.18.160.1
) is responding differently by temporarily caching the IP to github.com
.
172.18.160.1
is managed by WSL2 adding to /etc/resolv.conf
, while the Ubuntu WSL2 install it's interface assigned to an IP in the 172.18.0.0/20
subnet (range: 172.18.160.1 - 172.18.175.254
). So I assume the caching is going on there and differs from Windows 11 which connects directly to (EDIT: Nope seems to be cached at 192.168.1.1
👍192.168.1.1
)
The Windows 11 host likewise behaves similarly (with the additional error for 192.168.1.1
which represents the actual router device handling the DNS query for each host/wsl2/docker client):
$ ./doggo.exe --debug -x 4.237.22.38
time=2024-08-07T12:52:23.652+12:00 level=DEBUG msg="LoadNameservers: Initial nameservers" nameservers=[]
time=2024-08-07T12:52:23.653+12:00 level=DEBUG msg="No user specified nameservers, falling back to system nameservers"
time=2024-08-07T12:52:23.660+12:00 level=DEBUG msg="Loaded system nameservers" nameservers="[{Address:192.168.1.1:53 Type:udp} {Address:0.0.0.0:53 Type:udp}]"
time=2024-08-07T12:52:23.660+12:00 level=DEBUG msg="initiating UDP resolver"
time=2024-08-07T12:52:23.660+12:00 level=DEBUG msg="initiating UDP resolver"
time=2024-08-07T12:52:23.660+12:00 level=DEBUG msg="Attempting to resolve" domain=38.22.237.4.in-addr.arpa. ndots=0 nameserver=0.0.0.0:53
time=2024-08-07T12:52:23.660+12:00 level=DEBUG msg="Attempting to resolve" domain=38.22.237.4.in-addr.arpa. ndots=0 nameserver=192.168.1.1:53
time=2024-08-07T12:52:28.666+12:00 level=ERROR msg="Error looking up DNS records" error="context deadline exceeded"
But it can't seem to resolve the A
record either:
$ ./doggo.exe --debug A github.com
time=2024-08-07T12:54:54.242+12:00 level=DEBUG msg="LoadNameservers: Initial nameservers" nameservers=[]
time=2024-08-07T12:54:54.243+12:00 level=DEBUG msg="No user specified nameservers, falling back to system nameservers"
time=2024-08-07T12:54:54.248+12:00 level=DEBUG msg="Loaded system nameservers" nameservers="[{Address:192.168.1.1:53 Type:udp} {Address:0.0.0.0:53 Type:udp}]"
time=2024-08-07T12:54:54.248+12:00 level=DEBUG msg="initiating UDP resolver"
time=2024-08-07T12:54:54.248+12:00 level=DEBUG msg="initiating UDP resolver"
time=2024-08-07T12:54:54.249+12:00 level=DEBUG msg="Attempting to resolve" domain=github.com. ndots=0 nameserver=0.0.0.0:53
time=2024-08-07T12:54:54.249+12:00 level=DEBUG msg="Attempting to resolve" domain=github.com. ndots=0 nameserver=192.168.1.1:53
time=2024-08-07T12:54:59.260+12:00 level=ERROR msg="Error looking up DNS records" error="context deadline exceeded"
Presumably the 0.0.0.0
query (which no longer has any other DNS server to connect to with my CoreDNS container no longer running) is causing that failure?
By providing the router DNS nameserver explicitly we actually get a result returned:
$ ./doggo.exe --debug -x 4.237.22.38 '@192.168.1.1'
time=2024-08-07T12:58:41.878+12:00 level=DEBUG msg="LoadNameservers: Initial nameservers" nameservers=[192.168.1.1]
time=2024-08-07T12:58:41.878+12:00 level=DEBUG msg="Added nameserver" nameserver="{Address:192.168.1.1:53 Type:udp}"
time=2024-08-07T12:58:41.878+12:00 level=DEBUG msg="LoadNameservers: Final nameservers" nameservers="[{Address:192.168.1.1:53 Type:udp}]"
time=2024-08-07T12:58:41.878+12:00 level=DEBUG msg="initiating UDP resolver"
time=2024-08-07T12:58:41.878+12:00 level=DEBUG msg="Attempting to resolve" domain=38.22.237.4.in-addr.arpa. ndots=0 nameserver=192.168.1.1:53
NAME TYPE CLASS TTL ADDRESS NAMESERVER STATUS
22.237.4.in-addr.arpa. SOA IN 300s ns1-04.azure-dns.com. 192.168.1.1:53 NXDOMAIN
azuredns-hostmaster.microsoft.com.
1 3600 300 2419200 300
$ ./doggo.exe --debug A github.com '@192.168.1.1'
time=2024-08-07T12:56:14.792+12:00 level=DEBUG msg="LoadNameservers: Initial nameservers" nameservers=[192.168.1.1]
time=2024-08-07T12:56:14.793+12:00 level=DEBUG msg="Added nameserver" nameserver="{Address:192.168.1.1:53 Type:udp}"
time=2024-08-07T12:56:14.793+12:00 level=DEBUG msg="LoadNameservers: Final nameservers" nameservers="[{Address:192.168.1.1:53 Type:udp}]"
time=2024-08-07T12:56:14.793+12:00 level=DEBUG msg="initiating UDP resolver"
time=2024-08-07T12:56:14.794+12:00 level=DEBUG msg="Attempting to resolve" domain=github.com. ndots=0 nameserver=192.168.1.1:53
NAME TYPE CLASS TTL ADDRESS NAMESERVER
github.com. A IN 34s 4.237.22.38 192.168.1.1:53
That shares the same observation of caching A
for reverse DNS queries that was observed with WSL2, so that's all happening at the router 192.168.1.1
, not Windows host or WSL2 own nameserver IP.
Just sharing this error I came across (although it seems to be dependent on nameserver used for the query):
I'm aware of the local DNS service being a bit faulty (see details below), I'm not sure if this is something
doggo
can actually resolve either (affectsq
too, so I suppose both projects are handling the query in the same way?).dig
can resolve the record without issue in the same environment, thus this failure may be specific to Go (or a common DNS package, I haven't comparedq
anddoggo
packages or src).As can be seen below,
github.com
has a largeTXT
record that splits into multiple parts, I suspect that's the one related to the overflow error?q
has a option--txtconcat
to merge these into a single string for displayed output, but the error seems to suggest that this is a failure at the DNS name server used for the query not being able to handle this.My local DNS in this case is a bit complicated
/etc/resolv.conf
managed by Docker Desktop (192.168.65.0/24
network configured, Docker manages it's own internal DNS layer)./etc/resolv.conf
(172.x.y.z
nameserver), but Docker Desktop may actually run containers in a separate VM IIRC, rather than this Ubuntu 22.04 one that is provisioned by default and where I interface with the container via Windows Terminal. So this name server probably isn't in use, I know that if the container runs in host mode networking it's not reachable from my WSL2 Ubuntu 22.04 or Windows 11 host.1.1.1.1
configured for the nameserver to use, DNS should be going through this to reach the internet AFAIK, so it may be an issue with Dockers internally managed DNS layer and potentially whatever upstream it routes queries through.dig
/drill
also confirm that I cannot perform reverse DNS queries either, unless I provide an alternative name server to query like via@1.1.1.1
.@1.1.1.1
, but Docker containers do not return anything. I've also changed the Windows host network to use DHCP again instead of1.1.1.1
and there is no issue there either for Ubuntu.dig
has no trouble in a container however querying theTXT
record forgithub.com
.Here is the containers
/etc/resolv.conf
(comment is generated from the managed file contents by Docker):While the Windows host is configured with
1.1.1.1
, it wasn't when these issues started and the Docker daemon has not been restarted yet. It's also possible that the comment is misleading if the Docker host in this case is the WSL2 VM that Docker Desktop manages, where it may set a different nameserver (not sure how to access that). So I suspect either the issue is temporary, or the internal DNS layer Docker has is faulty.Via DNS server in
/etc/resolv.conf
(fail)For reference, same failure via
q
:Via
@1.1.1.1
(success):