Performance Testing / Benchmarking

[thesecretmaster]

This is an update on #15. As of https://github.com/thesecretmaster/predis/commit/cdd76b8da08dd9507b5690144054ece62dd6e1a6, we benchmark relatively similarly to redis-server for sequential workloads. The next step to testing is to attempt a benchmark for parallel workloads. This means modifying the output of gen_test_file.c to take a list of files to write to and a number of tests per file. It should also spit out every line to standard output. We will also need to move from redis-cli to some redis client c library, because we'll need the parallelism.

We should expect to outperform redis-server with this workload.

[thesecretmaster]

After trying again with a larger test data size, we're still a few times slower then redis. I noticed this using the tests from #15 again, and after digging in a bit I discovered that this is almost certainly due to cache misses. This is likely because of the hashtable updates which make the hashtable spit out full struct main_eles. Here's the proof:

$ perf stat -B -e cache-references,cache-misses,cycles,instructions,branches,faults,migrations redis-server --port 8080
25195:M 08 Nov 18:00:23.578 * Increased maximum number of open files to 10032 (it was originally set to 1024).
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 3.0.6 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 8080
 |    `-._   `._    /     _.-'    |     PID: 25195
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

25195:M 08 Nov 18:00:23.579 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
25195:M 08 Nov 18:00:23.579 # Server started, Redis version 3.0.6
25195:M 08 Nov 18:00:23.579 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
25195:M 08 Nov 18:00:23.580 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
25195:M 08 Nov 18:00:23.580 * The server is now ready to accept connections on port 8080
^C25195:signal-handler (1573257634) Received SIGINT scheduling shutdown...
25195:M 08 Nov 18:00:34.310 # User requested shutdown...
25195:M 08 Nov 18:00:34.310 # Redis is now ready to exit, bye bye...

 Performance counter stats for 'redis-server --port 8080':

       602,640,304      cache-references                                            
         4,391,304      cache-misses              #    0.729 % of all cache refs    
     7,412,605,027      cycles                                                      
     5,045,948,705      instructions              #    0.68  insns per cycle        
       969,255,802      branches                                                    
             4,346      faults                                                      
               203      migrations                                                  

      10.740577916 seconds time elapsed

$ perf stat -B -e cache-references,cache-misses,cycles,instructions,branches,faults,migrations bin/predis-server 
Accepted a conn 4!
Closed 4!
^C
 Performance counter stats for 'bin/predis-server':

       858,937,684      cache-references                                            
        98,429,109      cache-misses              #   11.459 % of all cache refs    
    17,739,363,371      cycles                                                      
     9,447,743,588      instructions              #    0.53  insns per cycle        
     1,879,046,351      branches                                                    
             4,220      faults                                                      
               113      migrations                                                  

      13.337431267 seconds time elapsed

[thesecretmaster]

Closing