blinkdb-js / blinkdb

🗃️ An in-memory JS database optimized for large scale storage on the frontend.
https://blinkdb.io
MIT License
119 stars 10 forks source link

Improve write performance for operations on tables with indexes + Bugfix for `remove` #3

Closed Valerionn closed 2 years ago

Valerionn commented 2 years ago

I have changed some create/remove operations to now edit the array in-memory instead of allocating a new array which should be a lot faster:

grafik

This should not change the behavior in any observable way (unless an index array contains the same item multiple times, which should not happen as far as I know) but provides a significant performance boost especially when inserting entries.

I added some benchmarks below, using a table with 1 index. However, the performance increase is even bigger when using multiple indexes (e.g. creating 10.000 items with 5 indexes takes 40ms now instead of 78ms, using clone in both cases).

Additionally, I fixed a small bug in the remove implementation that I stumbled upon.

Create 10.000 items:

Before

  blinkdb with index      took 24.31350ms, high 95.36340ms, low 22.92340ms
  blinkdb with no index   took 12.05970ms, high 106.57190ms, low 9.89640ms

After

  blinkdb with index      took 18.07080ms, high 74.83170ms, low 15.48300ms
  blinkdb with no index   took 11.05330ms, high 98.95320ms, low 10.08770ms

Update one item:

Before

  blinkdb with index      took 0.00510ms, high 1.11310ms, low 0.00200ms
  blinkdb with no index   took 0.00250ms, high 0.37020ms, low 0.00200ms

After

  blinkdb with index      took 0.00300ms, high 0.67560ms, low 0.00200ms
  blinkdb with no index   took 0.00220ms, high 0.99160ms, low 0.00190ms

Remove one item:

Before

No before available due to a bug in the current implementation which leads to faster execution time (but a wrong result)

After

  blinkdb with index      took 0.00580ms, high 2.15120ms, low 0.00270ms
  blinkdb with no index   took 0.00290ms, high 1.33420ms, low 0.00230ms
netlify[bot] commented 2 years ago

Deploy Preview for blinkdb canceled.

Name Link
Latest commit 3cfd2cdf6c72bd69a22428f6ea3eb8b54b774837
Latest deploy log https://app.netlify.com/sites/blinkdb/deploys/6320af1330a6f900090b0cec
maradotwebp commented 2 years ago

Thanks for creating a new benchmark - I'll eventually remove these files altogether, and build some benchmarks right into the docs so I can

  1. Test BlinkDB unter realistic circumstances (i.e. in the browser)
  2. Compare it to Redux or similar store alternatives (right now I'm only comparing performance to lokijs in order to make sure I'm not writing poorly performing code)
  3. Showcase the speed :P
maradotwebp commented 2 years ago

Not specific to your PR, but I noticed that it would probably be a good idea to throw errors if the DB reaches an inconsistent state (for example if an item is present in the primary btree, but not in other indexes). Performance wise, branch prediction will take care of it, and user wise