socketry / async-redis

MIT License
83 stars 18 forks source link

Add redis-rb with hiredis to the perf benchmarks #12

Closed davidor closed 5 years ago

davidor commented 5 years ago

I saw https://github.com/socketry/async-redis/pull/9 and thought that it would be interesting to add redis-rb with Hiredis to the bechmarks. https://github.com/redis/redis-rb#hiredis

Hiredis is implemented in C so the comparison is not entirely fair, but I think it's useful to know how it performs.

Results on my local machine:

Client Performance                                                                                                                                                              
Warming up --------------------------------------                                                                                                                               
 async-redis (pool)   780.000  i/100ms                                                                                                                                         
 async-redis (nested)   797.000  i/100ms                                                                                                                                         
 redis-rb     1.062k i/100ms                                                                                                                                         
 redis-rb (hiredis)     1.472k i/100ms                                                                                                                                         
Calculating -------------------------------------                                                                                                                               
  async-redis (pool)      8.222k (±10.0%) i/s -     41.340k in   5.077386s                                                                                                      
  async-redis (nested)      9.353k (± 9.6%) i/s -     47.023k in   5.073603s                                                                                                      
  redis-rb     10.760k (± 9.5%) i/s -     54.162k in   5.078243s                                                                                                      
  redis-rb (hiredis)     14.904k (± 5.9%) i/s -     75.072k in   5.054911s                                                                                                      

Comparison:                                                                                                                                                                     
  redis-rb (hiredis):    14903.9 i/s                                                                                                                                            
  redis-rb:    10759.8 i/s - 1.39x  slower                                                                                                                            
  async-redis (nested):     9353.0 i/s - 1.59x  slower                                                                                                                            
  async-redis (pool):     8221.5 i/s - 1.81x  slower   

I think that in the future we could add more redis commands. We could select the commands to test based on what the redis-benchmark tool tests: https://redis.io/topics/benchmarks

coveralls commented 5 years ago

Pull Request Test Coverage Report for Build 94


Totals Coverage Status
Change from base Build 90: 0.0%
Covered Lines: 341
Relevant Lines: 394

💛 - Coveralls
davidor commented 5 years ago

I noticed that Travis also runs the tests on TruffleRuby and JRuby. Those fail because they do not support C extensions.

I'll try to provide a fix for that tomorrow.

ioquatix commented 5 years ago

Okay, once you fix the test failures we can merge.

davidor commented 5 years ago

I think that the issue should be fixed.

I see that JRuby tests are still failing but it's because of #13 . TruffleRuby is failing as well, but it looks like it's not because of something introduced in this PR, because my other PR fails too in TruffleRuby #14 with the same error.

ioquatix commented 5 years ago

Okay let's see how it goes.