Results of iterator micro optimization

My contractor gave me some time to test and integrate RocksDB with go etc. Just for those interested in and playing with optimization: I added some functions for the iterator which assume to reduce cgo calls, for example: NextValidKeyValue(), which calls a c function that calls the iterator`s next and if valid returns the key and value:

iter.SeekToFirst()
for valid, skey, sval := iter.ValidKeyValue(); valid; valid, skey, sval = iter.NextValidKeyValue() {}

Results:

Original implementation:
(pprof) top 30
20.96s of 21.80s total (96.15%)
Dropped 94 nodes (cum <= 0.11s)
Showing top 30 nodes out of 46 (cum >= 0.16s)
      flat  flat%   sum%        cum   cum%
    12.89s 59.13% 59.13%     13.85s 63.53%  runtime.cgocall
     2.27s 10.41% 69.54%      3.86s 17.71%  runtime.mallocgc
     0.96s  4.40% 73.94%      0.96s  4.40%  runtime.heapBitsSetType
     0.60s  2.75% 76.70%      1.01s  4.63%  runtime.cgoIsGoPointer
     0.49s  2.25% 78.94%      4.35s 19.95%  runtime.newobject

Iterator extension: 
(pprof) top 30
17.37s of 18.15s total (95.70%)
Dropped 77 nodes (cum <= 0.09s)
Showing top 30 nodes out of 46 (cum >= 0.24s)
      flat  flat%   sum%        cum   cum%
     7.20s 39.67% 39.67%      7.55s 41.60%  runtime.cgocall
     4.50s 24.79% 64.46%      5.99s 33.00%  runtime.mallocgc
     0.83s  4.57% 69.04%      0.83s  4.57%  runtime.heapBitsSetType
     0.72s  3.97% 73.00%      0.90s  4.96%  runtime.cgoIsGoPointer
     0.68s  3.75% 76.75%     17.19s 94.71%  github.com/kapitan-k/gorocksdborig.(*Iterator).NextValidKeyValue

Not that of an advantage.

Iterator and tests can be found in my fork: https://github.com/kapitan-k/gorocksdb/tree/iterator_perf

tecbot / gorocksdb

Results of iterator micro optimization #118