[Proposal] On HashRedisStore, when loading coverage, get the latest report from a cache and defer current report generation to a background thread

[Drowze]

Running Coverband with HashRedisStore with a significantly big project (~4600 files) has proven difficult, as everytime we try to load the coverage page, Coverband does a lot of Redis HGETALL commands (2 per file) and blocks the request until the commands return.

This PR aims to mitigate the hit HashRedisStore has on Redis when generating the coverage report by introducing a caching layer to, so the HGETALL commands used to build the coverage report run inside background threads, in batches of a 250 (per file type, concurrently). All following requests will then receive the latest cache result, while a new cache is re-populated in the background.

This optimization improved load times of the report page from 30-40 seconds to 15-20 seconds (on a 2GB redis server with about 40% memory left). Tested with:

Benchmark.measure do
  Coverband::Reporters::Web.new.tap do |cov|
    cov.instance_variable_set(:@request,Rack::Request.new(Rack::MockRequest.env_for("/")))
  end.index
  nil
end

# worst case i.e. without any cache
# => #<Benchmark::Tms:0x00007f082f4ddee0 @cstime=0.0, @cutime=0.0, @label="", @real=36.507868222999605, @stime=2.697597, @total=36.615393, @utime=33.917795999999996>

# with a cache
# => #<Benchmark::Tms:0x00007f081aaf1558 @cstime=0.0, @cutime=0.0, @label="", @real=16.935891766999703, @stime=0.9192540000000005, @total=16.973998, @utime=16.054744>

[danmayer]

ouch this one definitely adds a lot of complexity but perhaps the background thread and iterative caching is simpler than the approach I was considering with paging in the data... Since this is opt in and off by default, I feel like I can pull it in and we can get some feedback about how it performs for various folks that have been hitting performance issues.

[danmayer]

changes are in the 6.0.2 release, thanks so much for your contributions