Resolv::ResolvError for hosts resolved by /etc/resolver/* files on OS X

[gevans]

I'm experiencing inconsistent DNS resolution between initializing a new Celluloid::IO::TCPSocket and initializing a regular TCPSocket:

TCPSocket.new 'project.dev', 80
# => #<TCPSocket:fd 7>

Celluloid::IO::TCPSocket.new 'project.dev', 80
# Resolv::ResolvError: DNS result has no information for project.dev
# from /opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/celluloid-io-0.16.0.pre/lib/celluloid/io/tcp_socket.rb:67:in `initialize'

I'm not entirely certain what the difference in host resolution is between the two implementations. Socket.gethostbyname works fine while Resolve.getaddress fails similarly to Celluloid::IO::DNSResolver#resolve.

Boxen uses dnsmasq to set up a *.dev wildcard that resolves to 127.0.0.1. The interesting thing however, is that the nameserver is specified in /etc/resolver/dev, what appears to be an OS X-specific format where the name of the file is the TLD and the contents is a regular /etc/resolv.conf definition:

$ cat /etc/resolv.conf
nameserver 10.0.1.1 # a nameserver that does not resolve *.dev hosts

$ cat /etc/resolver/dev
nameserver 127.0.0.1 # a nameserver that *does* resolve *.dev hosts

Anyway, I'm not sure what the project's stance is in regard to supporting odd platform-specific setups (there's only enough room for Windows to do that!) but I'm curious what the reason is for resolution to be working fine everywhere else.

Thanks!

[tarcieri]

Celluloid uses its own asynchronous DNS resolver to prevent DNS requests from blocking the Celluloid::IO event loop.

My first advice would be to try master. We just merged some changes to the DNS resolver in f156afb1ec9a2d67ac06df23e96a30d2a63ab339 that might address your problem. This should, hopefully, use Ruby's own Resolv to do the local hostname lookups.

[gevans]

I gave it a go and the results were the same. My original post was a bit lengthy so to summarize:

• Resolve and Celluloid::IO::DNSResolver behave similarly. Neither can resolve the hostname.
• TCPSocket.new and Celluloid::IO::TCPSocket.new differ. Ruby's TCPSocket can resolve the hostname while Celluloid::IO cannot.

I looked into it more and it looks like TCPSocket uses Socket.getaddrinfo which then calls the C function getaddrinfo. That explains a lot.

I noticed pull #65 was an attempt to fix some platform-specific issues with DNS resolution. Is there still an effort to continue in this direction?

[Asmod4n]

You could make dnsmasq your default resolver and let it forward all queries it can't answer to the other dns server

Von einem mobilen Gerät gesendet

Am 03.05.2014 um 01:53 schrieb Gabe Evans notifications@github.com:

I gave it a go and the results were the same. My original post was a bit lengthy so to summarize:

Resolve and Celluloid::IO::DNSResolver behave similarly. Neither can resolve the hostname. TCPSocket.new and Celluloid::IO::TCPSocket.new differ. Ruby's TCPSocket can resolve the hostname while Celluloid::IO cannot. I looked into it more and it looks like TCPSocket uses Socket.getaddrinfo which then calls the C function getaddrinfo. That explains a lot.

I noticed pull #65 was an attempt to fix some platform-specific issues with DNS resolution. Is there still an effort to continue in this direction?

— Reply to this email directly or view it on GitHub.

[gevans]

@Asmod4n That is indeed a temporary workaround I've been using.

[digitalextremist]

@gevans is this still an issue for you, or has your workaround been enough?

[gevans]

After rereading my original post, I realize I wasn't very clear on what my issue was. The issue I have is that Celluloid::IO::TCPSocket explicitly resolves hostnames through DNS with its own resolver which differs greatly from the way the standard TCPSocket is implemented. This results in unexpected and hard to debug issues (host example.com works in a console, TCPSocket works in MRI, but Celluloid fails completely).

It assumes DNS servers are only specified in /etc/resolv.conf and ignores other common configurations such as /etc/hosts.

As an example, Linux (glibc) has /etc/nsswitch.conf which provides many different methods for resolution. Some distributions like CoreOS use this facility to handle resolution through additional discovery mechanisms. Avahi (Bonjour) also uses this to allow *.local resolution. OS X has its own methods and I'm sure Windows does as well.

This isn't normally a problem because the C library of each platform handles these specifics for the developer.

This difference should either be fixed or documented somewhere. Admittedly, I'm no longer using Celluloid::IO so this is no longer my issue. Please feel free to close at your own discretion. :smile:

[digitalextremist]

@gevans thanks for the in-depth description of the situation even though you're no longer using Celluloid::IO!

[Asmod4n]

libuv has a async dns resolver build in, which libev is missing; I don't know how Java solves the blocking issues.

So this is more likely a nio4r issue.

[tarcieri]

It assumes DNS servers are only specified in /etc/resolv.conf and ignores other common configurations such as /etc/hosts.

@gevans what? No, that's completely incorrect. The code for handling things like /etc/hosts in a cross-platform way is here:

https://github.com/celluloid/celluloid-io/blob/master/lib/celluloid/io/dns_resolver.rb#L36 https://github.com/celluloid/celluloid-io/blob/master/lib/celluloid/io/dns_resolver.rb#L64 https://github.com/celluloid/celluloid-io/blob/master/lib/celluloid/io/dns_resolver.rb#L68

[gevans]

@tarcieri My bad. Ignore that statement.

[tarcieri]

I'll admit Celluloid::IO doesn't have the world's greatest DNS resolver and would be happy to consider alternatives

[ioquatix]

RubyDNS has an awesome DNS resolver :)

[digitalextremist]

@ioquatix can you abstract your resolver and provide it as a stand-alone gem?

[tarcieri]

@ioquatix it'd be nice if you could use RubyDNS's resolver with Celluloid::IO

[ioquatix]

It should be trivial to extract into a gem.