Query Optimization

[Eunovo]

Feature Request

Is your feature request related to a problem? Please describe. Currently, DataStore's Query filter is not optimal. It doesn't use any index available and uses O(n) search time always

Describe the solution you'd like DataStore Query filter should make use of indices where possible

@wtrocki @kingsleyzissou Please state any other issue you have with DataStore querying here

[Eunovo]

@wtrocki @kingsleyzissou I reopened this issue. I didn't realize it had been closed. I changed the title to more clearly reflect the topic.

[Eunovo]

@wtrocki @kingsleyzissou There are two major things I'm going to do to allow predicates use indexes (IDBKeyRange for Indexeddb);

• Add field index information to predicate definitions; currently, the predicates don't know if they contain any indexes
• Create a platform-specific(in this case IndexedDB) execution function to execute predicates Any objections or contributions?

[wtrocki]

Let's chat about it first. We need more details.

Add field index information to predicate definitions; currently, the predicates don't know if they contain any indexes

What do you mean by this? I'm not sure if we should extend existing predicates.

Create a platform-specific(in this case IndexedDB) execution function to execute predicates

-1. Any reason for platform-specific?

[Eunovo]

-1. Any reason for platform-specific?

Indexes work differently on different platforms. We have to executor function for each platform that makes use of that platform's Indexes. For example, IndexedDB uses key ranges but Sqlite doesn't.

[Eunovo]

What do you mean by this? I'm not sure if we should extend existing predicates.

I'm not really extending them. I just want to add index information so the execution function can find indexes in predicates

[wtrocki]

@Eunovo What is the outcome of this? Did we get any benefits? Anything else to be done.

As far I know we have support for indexes in both providers for the moment. Question is is it possible to utilize those from query side?

[Eunovo]

Question is is it possible to utilize those from query side?

I'm not sure how we will do this on the IDB side, but it's probably possible on the SQL side. Last time I tried to this on IDB, it proved to be difficult.

We do have queryById now which takes advantage of the primary key index to find documents.

[wtrocki]

Having and queries utilizing indexes will be possible. This task is blocked until we get solid support for the both db providers.