Kong / lua-resty-dns-client

Lua DNS client, load balancer, and utility library
http://kong.github.io/lua-resty-dns-client/topics/README.md.html
Apache License 2.0
152 stars 52 forks source link

DNS query does not do any retries when dns server returned error #66

Closed olderwei closed 4 years ago

olderwei commented 5 years ago

I configured three nameserver in /etc/resolv.conf file. When DNS Server is requested, an error is returned similar to ServFail, lua-resty-dns-client and lua-resty-dns will not retry other DNS Server,Is there a specific reason behind this?

olderwei commented 5 years ago

@Tieske

Tieske commented 5 years ago

what's your resolv.conf, and what is your code?

The resolver should drop the configuration in the logs when initialized, can you show those logs?

Tieske commented 5 years ago

Log level must be debug, retries for example: https://github.com/Kong/lua-resty-dns-client/blob/master/src/resty/dns/client.lua#L583

olderwei commented 5 years ago

The log is as follows: 2019/04/30 10:08:05 [debug] 19566#0: [lua] client.lua:571: init(): [dns-client] nameserver 180.76.76.76 2019/04/30 10:08:05 [debug] 19566#0: [lua] client.lua:571: init(): [dns-client] nameserver 219.141.136.10 2019/04/30 10:08:05 [debug] 19566#0: [lua] client.lua:571: init(): [dns-client] nameserver 223.5.5.5 2019/04/30 10:08:05 [debug] 19566#0: [lua] client.lua:576: init(): [dns-client] attempts = 5

The kong's error log is as follows: 2019/04/26 21:16:10 [error] 141470#0: *373038045 [lua] balancer.lua:808: execute(): [dns] dns server error: 2 server failure. Tried: (short)..com:(na) - cache-miss ..com:1 - cache-hit/dns server error: 2 server failure ..com:33 - cache-hit/dns server error: 2 server failure ..com:5 - cache-hit/dns server error: 2 server failure , client: 172...*, server: kong, request: "GET /*** HTTP/1.1", host: ".***.com"

I just saw this pr, which is similar to what I want to express. https://github.com/openresty/lua-resty-dns/pull/9

I wonder if the following code can be modified. https://github.com/openresty/lua-resty-dns/blob/bb21982a51cfdab22a8c3bd4a37f92b1c56aa43d/lib/resty/dns/resolver.lua#L885 ` if err and err ~= "id mismatch" then break end

if answers.errcode then break end

if answers then return answers, nil, tries end `

olderwei commented 5 years ago

@Tieske

Tieske commented 5 years ago

That is a weird log, did you redact the logs to remove the hostname? the name tried is "..com" which doesn't resolve.

Additionally the log snippet you posted above contains - cache-hit/dns server error: 2 server failure which means that you're hitting the cached error. This is not the initial error. Note that errors are also cached, see https://docs.konghq.com/1.1.x/configuration/#dns_error_ttl

What upstream hostname are you trying to resolve? And what is the initial DNS error?

Tieske commented 5 years ago

btw: in the error message 2 server failure actually means that the resolver reached out to the DNS server, and the DNS server responded with an error code 2, which indicates a server failure.

Essentially this means that on Kong side everything works as designed. So my guess is that you mis configured your upstream hostname since it has ..com, which is not a valid hostname.

Tieske commented 5 years ago

@olderwei were you able to resolve this?

Another thing that comes to mind: when the UDP reply gets truncated (answer too big), the resolver will fall back on a TCP query. Now this TCP query does not handle retries. See also https://github.com/openresty/lua-resty-dns/issues/40

Tieske commented 5 years ago

@olderwei any update?

Tieske commented 4 years ago

closing due to no response