Kong / lua-resty-dns-client

Lua DNS client, load balancer, and utility library
http://kong.github.io/lua-resty-dns-client/topics/README.md.html
Apache License 2.0
152 stars 52 forks source link

perf(client) reduce amount of timers on init_worker #130

Closed ghost closed 3 years ago

ghost commented 3 years ago

We end up scheduling 1 timer for each hostname on the init_worker phase. To fix it we are "deduplicating" timers created to resolve the same hostname for the asynchronous case. That is, we are reusing timers for DNS queries of the same hostname. If we have 6k targets, and all of the targets are "httpbin.org" there's no reason to create 6k timers per worker. Instead, we should just create a maximum of 3 timers for 3 DNS queries (default: 1 for SRV, 1 for A and 1 for CNAME) and don't schedule more timers than that to resolve the same hostname. We should only schedule more timers if we try to resolve different hostnames. And the approach we're using in the patch is: if the current nginx phase can't be blocked by a semaphore (which is the case of init_worker context, but not of timer context, for example), then we should not schedule more timers for the same hostname, we should just return the previous query so that the previous timers can be reutilized (in the init_worker phase, for example). Otherwise, if the current nginx phase can blocked by a semaphore, then we will just block and wait for the DNS query to complete (in the timer context, for example, and this hasn't changed). In a nutshell: we are reutilizing timers scheduled to resolve the same hostname.

CLAassistant commented 3 years ago

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.