Review performance regressions.

[ioquatix]

We should be both faster than Bind9 and native resolver. Check why this has regressed.

[ioquatix]

I think I found the cause of the problem. There is an excessive amount of logging due to $DEBUG being set in Travis. For these performance specs, perhaps it should be disabled.

[tarcieri]

Travis is generally an unreliable place to run benchmarks

[ioquatix]

I appreciate that feedback.

• It's a relative benchmark so it's comparing two things. It's not the absolute performance, but the relative performance that indicates a regression and it's a useful check.

• I've been running these kinds of performance checks for a long time. While I don't expect travis to be fast, I've never had any specific issue with their performance.

The problem here is the use of $DEBUG to determine a default log level here:

https://github.com/socketry/async/blob/master/lib/async/logger.rb

I think it's not a bad choice except it's causing issues in this particular case. The solution is to disable it for the performance specs e.g. before(:all) { Async.logger.level = Logger::WARN }and the appropriate one afterwards, but there is more than one performance spec, so a shared context would be good here. I actually have a shared context in another project which includes support for Benchmark, RubyProf and some other basic profiling tools, it's really easy to use. I'll probably copy it across.