Level / memdown

In-memory abstract-leveldown store for Node.js and browsers.
MIT License
287 stars 37 forks source link

Add basic insert/delete benchmarks #8

Closed mhart closed 10 years ago

mhart commented 10 years ago

These don’t actually benchmark MemDOWN directly, but rather compare different strategies for inserting/deleting keys. (I'm not exactly sure as to the best strategy for comparing different versions of MemDOWN - just keep a running tally of benchmark results I guess?)

I figured bench was probably a bit more lightweight than any of the other options (less deps). Running these benchmarks on my MacBook Air I get:

Scores: (bigger is better)

binary insert
Raw:
 > 0.2933411557641537
 > 0.3137747097583935
 > 0.31446540880503143
 > 0.3082614056720099
Average (mean) 0.30746066999989713

push() + sort()
Raw:
 > 0.0022455308322610924
 > 0.0023219257123087604
 > 0.0023006911276147353
 > 0.002307960386167932
Average (mean) 0.00229402701458813

Winner: binary insert
Compared with next highest (push() + sort()), it's:
99.25% faster
134.03 times as fast
2.13 order(s) of magnitude faster
A LOT FASTER

and

Scores: (bigger is better)

binary delete
Raw:
 > 0.5425935973955507
 > 0.5339028296849974
 > 0.5136106831022085
 > 0.4975124378109453
Average (mean) 0.5219048869984254

linear delete
Raw:
 > 0.15087507543753773
 > 0.14490653528474134
 > 0.14575134819997085
 > 0.14863258026159334
Average (mean) 0.1475413847959608

Winner: binary delete
Compared with next highest (linear delete), it's:
71.73% faster
3.54 times as fast
0.55 order(s) of magnitude faster
QUITE A BIT FASTER
mhart commented 10 years ago

I didn't want to commit this directly on master in case you guys had some comments about whether this is even useful...?

rvagg commented 10 years ago

lgtm except for the .npmignore, I'd rather distribute with tests and benchmarks, they don't take up enough size to warrant removal; aside from that this can be merged I reckon

mhart commented 10 years ago

Ah ok - that was probably my bias creeping in against anything extra going into npm due to the pain we feel in one of our apps that has a dependency tree with over 2,000 (sub)modules. Esp with us being in Aus, every little byte counts!

Have removed that npmignore commit.