search

w3c / IndexedDB

Indexed Database API
https://w3c.github.io/IndexedDB/
Other
240 stars 62 forks source link

Batch get/getAll #376

Open cyanfish opened 2 years ago

cyanfish commented 2 years ago

Currently the only way to query a set of individual rows by key is to use repeated calls to get or getAll. We have some use cases where we want to query many keys with one row each (e.g. the equivalent of SQL WHERE id IN (1,3,9,...)), where the event loop potentially adds quite a bit of overhead.

Another option would be to add a batch method for get and/or getAll, for example:

request = store.batchGetAll(keyRangeArray) 
request = index.batchGetAll(keyRangeArray)

Which would return an array of arrays corresponding to the input array.

The Chrome team did some internal prototyping and found that in the best case (1 row per key) there is potential for a 3x speedup over mutliple parallel getAll calls. That hasn't been proven in real world use cases yet and it's unclear if other browsers would see the same improvements, but that would be a significant speedup that can't be replicated with the current APIs.

tophf commented 2 years ago

there is potential for a 3x speedup over mutliple parallel getAll calls

Is the base version using a single transaction with three getAll requests or three separate transactions?

cyanfish commented 2 years ago

The test I'm referring to used a single transaction with N getAll requests in parallel vs 1 prototype batchGetAll call with N ranges. N=100 to N=100k were tested.