socketry / async-dns

An asynchronous DNS resolver and server.
MIT License
96 stars 14 forks source link

DNS queries over TCP create a new socket per query, exhausting the available file descriptors #20

Open postmodern opened 2 years ago

postmodern commented 2 years ago

I noticed when performing many queries over TCP (ex: [:tcp, '8.8.8.8', 53]) that the ruby process would create a new TCP socket per-query. These TCP sockets would not be closed fast enough (ex; waiting in TIME_WAIT), and would eventually cause the process to hit it's maximum number of open file descriptors (ex: Errno::EADDRNOTAVAIL).

Steps To Reproduce

  1. ruby test.rb
  2. In another terminal run watch "netstat -n" to observe connections

Gemfile

source 'https://rubygems.org/'

gem 'async-dns'

test.rb

require 'bundler/setup'
require 'async/dns'

nameservers = [[:tcp, '8.8.8.8', 53]]
resolver = Async::DNS::Resolver.new(nameservers)
hostname = 'google.com'

Async do
  loop do
    begin
      addresses = resolver.addresses_for(hostname)
    rescue Async::DNS::ResolutionFailure
    end
  end
end

Note: if :tcp is changed to :udp, then only one socket is created and reused.

ioquatix commented 2 years ago

This gem doesn't use a pool, we should update it - it's based on a 10 year old code base which didn't have access to such a abstraction at the time.

postmodern commented 2 years ago

I was looking through the code and it appears that UDP "sockets" are reused, where as the TCP "sockets" are used to create a new Transport for each request. https://github.com/socketry/async-dns/blob/075264ba6bd69dac4738323b4dd083a825589e28/lib/async/dns/resolver.rb#L214-L230

ioquatix commented 2 years ago

I will sort this out but it won't be this month probably, I have a lot on my plate, but I'll clear some time to review this during February.