Performance optimization: make PREFERRED_TRIM_SIZE configurable

[frontend-sensei]

Why

I faced issue that if my initial YDoc starts from 3-5Mb this can lead to serious memory usage by IndexedDB on each update.

First steps

I investigated a lot of discussions, issues, forks etc. Understood each situation in detail.

What a problem?

People wait that this just should works good. But everyone has their own expectations. I think each tool need more detailed documentation about such processes.

After bunch hours of learning code and other situations I realized that inside y-indexeddb exists logic for optimization memory usage. It triggers under certain condition. You just need to reach it. But not so simple... 🫤

PREFERRED_TRIM_SIZE

On each document update this constant used for checking whether IndexedDB exceed to this limit. If yes - do trim. Trimming - reduce data usage of IndexedDB.

• PREFERRED_TRIM_SIZE equals 500
• _dbsize equals records count in IndexedDB.

By simple words:

if (idbPersistence._dbsize >= PREFERRED_TRIM_SIZE) {
  // do trim
}

So, for triggering data trimming I need to do 500 YDoc updates to update IndexedDb 500 times respectively.

Okay, whats next?

If your update size is 12b it really doesn't matter - 12b 500 = 6000b - 6kb But if your update is 4Mb here is interesting - 4Mb 500 = 2000Mb - 2Gb 🥲

And yeah, this is expected behavior.

So what do you want man? 🤔

I think the best approach would be to make PREFERRED_TRIM_SIZE configurable. For example decrease it value from 500 to 100. (4Mb*100 = 400Mb) Main idea that each can configure PREFERRED_TRIM_SIZE by their data size. If you know that your updates big - you decrease PREFERRED_TRIM_SIZE for triggering trimming earlier.

So this PR:

• Makes PREFERRED_TRIM_SIZE configurable from IndexeddbPersistence instance via passing options object

Example:

new IndexeddbPersistence("my-document-id", doc, {
    PREFERRED_TRIM_SIZE: 100
})

@dmonad, I would be glad if you can provide feedback for my contribution!

Peace ✌️

[frontend-sensei]

Feel free to ping me, if something can be improved 😉

[frontend-sensei]

@dmonad It's ready for your review

[frontend-sensei]

@dmonad Any updates?

[frontend-sensei]

@dmonad By any chance, do you have an update?