m13253 / dns-over-https

High performance DNS over HTTPS client & server
https://developers.google.com/speed/public-dns/docs/dns-over-https
MIT License
1.96k stars 221 forks source link

json-dns: Record data contains newline #122

Closed brotaxt closed 2 years ago

brotaxt commented 2 years ago

When using [[upstream.upstream_google]] and activating EDNS Client Subnet in dnsmasq. I get tons of Record data contains newline messages from doh-client.

doh-client[4535]: 2022/02/02 09:08:10 json-dns: Record data contains newline: "\n;; OPT PSEUDOSECTION:\n; EDNS: version 0; flags: ; udp: 1232"
doh-client[4535]: 2022/02/02 09:08:10 json-dns: Record data contains newline: "\n;; OPT PSEUDOSECTION:\n; EDNS: version 0; flags: ; udp: 1232"
doh-client[4535]: 2022/02/02 09:08:10 json-dns: Record data contains newline: "\n;; OPT PSEUDOSECTION:\n; EDNS: version 0; flags: ; udp: 1232"
doh-client[4535]: 2022/02/02 09:08:20 json-dns: Record data contains newline: "\n;; OPT PSEUDOSECTION:\n; EDNS: version 0; flags: ; udp: 1232"
doh-client[4535]: 2022/02/02 09:08:20 json-dns: Record data contains newline: "\n;; OPT PSEUDOSECTION:\n; EDNS: version 0; flags: ; udp: 1232"
doh-client[4535]: 2022/02/02 09:08:27 json-dns: Record data contains newline: "\n;; OPT PSEUDOSECTION:\n; EDNS: version 0; flags: ; udp: 1232"
doh-client[4535]: 2022/02/02 09:08:32 json-dns: Record data contains newline: "\n;; OPT PSEUDOSECTION:\n; EDNS: version 0; flags: ; udp: 1232"

After changing the upstream to [[upstream.upstream_ietf]] the messages doesn't appear anymore.

Is this an expected behaviour?

Greetings!

brotaxt commented 2 years ago

i'm experiencing some really weird issues with youtube.de / youtube.com when using [[upstream.upstream_ietf]]

One Device in my network cannot access youtube, when using IETF. The Website shows "you are offline" and after refreshing the Website I get "nxdomain". However nslookup on youtube.com and youtube.de works fine. The dnsmasq-logs doesn't show anything obvious. All queried domains are getting resolved.

When I change the doh-client config to[[upstream.upstream_google]] youtube works as expected.

Iam using a local dnsmasq, which forwards my dns requests to another dnsmasq on a VPS which in turn uses the doh-client to resolve the dns queries.

m13253 commented 2 years ago

IETF and Google are totally two different protocols. If dnsmasq only supports one protocol, you should only use that protocol.

Furthermore, uploading your configuration file and detailed log (enable verbose) would be helpful in diagnosing the problem.

brotaxt commented 2 years ago

dnsmasq only forwards the requests to the doh-client. Iam surprised about two things.

my local dnsmasq.conf

log-async
no-resolv
cache-size=10000
domain-needed
bogus-priv
no-dhcp-interface=eth0
no-dhcp-interface=docker0
no-dhcp-interface=lo
filterwin2k
dns-forward-max=1000
no-hosts
no-poll

# Activate ECS/EDNS0 Client Subnet
add-subnet=MY_LOCAL_IPv4_SUBNET/32,MY_LOCAL_IPv6_SUBNET/128

server=XXX.XXX.XXX.XXX (IPv4)
server=XXX.XXX.XXX.XXX (IPv6)

doh-client.conf (on my VPS)

listen = [
    "127.0.0.1:5380",
    "[::1]:5380",

]

[upstream]

upstream_selector = "random"

[[upstream.upstream_google]]
    url = "https://dns.nextdns.io/a6dxxx"
    weight = 100

[others]
bootstrap = [

    "8.8.8.8:53",
    "8.8.4.4:53",

]

passthrough = [
]

timeout = 30
no_cookies = true
no_ecs = false
no_ipv6 = false
no_user_agent = false
verbose = false
insecure_tls_skip_verify = false

/etc/dnsmasq.conf (on my VPS)

localise-queries
no-resolv

cache-size=10000

server=127.0.0.1#5380
server=::1#5380

log-async
domain-needed
expand-hosts
bogus-priv
interface=eth0

doh-client.log

Feb 05 01:17:54 localhost doh-client[26781]: [::1]:57351 - - [05/Feb/2022:01:17:54 +0100] "1.pool.ntp.senec-ies.com. IN A"
Feb 05 01:17:54 localhost doh-client[26781]: [::1]:46542 - - [05/Feb/2022:01:17:54 +0100] "1.pool.ntp.senec-ies.com. IN AAAA"
Feb 05 01:17:54 localhost doh-client[26781]: 2022/02/05 01:17:54 choose upstream: upstream type: Google, upstream url: https://upstream-dns-url
Feb 05 01:17:54 localhost doh-client[26781]: 2022/02/05 01:17:54 json-dns: Record data contains newline: "\n;; OPT PSEUDOSECTION:\n; EDNS: version 0; flags: ; udp: 1232"
Feb 05 01:17:54 localhost doh-client[26781]: 2022/02/05 01:17:54 json-dns: Record data contains newline: "\n;; OPT PSEUDOSECTION:\n; EDNS: version 0; flags: ; udp: 1232"
m13253 commented 2 years ago

I can reproduce your results. Here is my conclusion:

  1. NextDNS supports both IETF and Google mode. Both mode should work.
  2. The “Record data contains newline” message is actually only a warning, instead of an error. It will drop the affected record (which is not important) and continue, without affecting the whole DNS resolution.
  3. While trying to resolve YouTube's IP address with ECS on, NextDNS waited for a couple of seconds before returning the results.

Therefore, I would suggest you to increase the timeout duration and/or retry interval in dnsmasq (both home and VPS side). Sometimes dnsmasq is not patient enough to wait for the result to come in.

You can also use the command dig to diagnose DNS problems:

$ dig -p5380 @127.0.0.1 www.youtube.com +dnssec +subnet=1.1.1.0/24

At least on my side I can successfully resolve YouTube's IP address.