w3c / IndexedDB

Indexed Database API
https://w3c.github.io/IndexedDB/
Other
245 stars 62 forks source link

Support N-dimensional queries on indexes #40

Open inexorabletash opened 9 years ago

inexorabletash commented 9 years ago

Short summary, if an index's array keypath represents multiple dimensions, using an IDBKeyRange.bound([minX, minY, ...], [maxX, maxY, ...]) doesn't behave as some developers expect. This has come up a few times on Stack Overflow:

http://stackoverflow.com/questions/21731347/indexeddb-idbkeyrange-compound-multiple-index-not-working

http://stackoverflow.com/questions/16522115/indexed-db-cursor-ranges-on-mulitiple-properties

This is because IDBKeyRange represents a one-dimensional range.

A general N-dimensional query using cursors is plausible but subtle:

https://gist.github.com/inexorabletash/704e9688f99ac12dd336

We could support this scenario with another query object type, array of ranges, or ...?

inexorabletash commented 9 years ago

Another one: http://stackoverflow.com/questions/12084177/in-indexeddb-is-there-a-way-to-make-a-sorted-compound-query

inexorabletash commented 8 years ago

Again: http://stackoverflow.com/questions/35040442/javascript-indexeddb-searching-a-range-of-value-using-multiple-keys

inexorabletash commented 8 years ago

And: http://stackoverflow.com/questions/37302019/why-i-cannot-get-the-correct-result-through-idbkeyrange-bound-function-in-indexe

inexorabletash commented 8 years ago

Although the discoverability is low, I'm intrigued by:

index.openCursor([IDBKeyRange.bound(lo1, hi1), IDBKeyRange.bound(lo2, hi2), ...])

inexorabletash commented 8 years ago

Here's another variation:

http://stackoverflow.com/questions/37807022/how-to-do-select-from-table-where-indexa-a-order-by-indexb-asc-limit-10

In this case the user wants SELECT ... WHERE indexA >= 'a' ORDER BY indexB ASC LIMIT 10

This can be done with an N-dimensional select (with the syntax proposed above) by:

let index = store.createIndex('by_b_a', ['indexB', 'indexA']);
index.getAll([null, IDBKeyRange.lowerBound('a')], 10);

... but that's still basically a "full table scan". Any DB experts know what primitives should exist to satisfy this?

nolanlawson commented 8 years ago

I agree this is a problem, but I would like to have a general-purpose API to query across multiple indexes, rather than limiting this to a single index. E.g. consider SELECT * WHERE age > 30 && lastName === 'Lawson'.

This would have the benefit of solving a further existing problem with IDB indexes, which is that (in the case of the above example), either age or lastName would have to be done in-memory in JavaScript. (FWIW this is actually a problem we have in PouchDB right now, with the allDocs() API as described in #92.) Sorry, just realized the issue is about something else; this is already largely covered in #45 and #92.

nolanlawson commented 8 years ago

I think your proposed API definitely improves the ergonomics of the existing one, but I also agree that it's a bit non-obvious. I believe my proposal in https://github.com/w3c/IndexedDB/issues/45#issuecomment-248869717 would also cover this use case (although it's much more ambitious, so the change you proposed could be a nice quick win).