Question: Does Sembast support indexing?

[isaacfi]

Hi,

This DB is what I was looking for, but there are not documentation about performance or the posibility of index some properties. In my app, I have to store around of 500, 000 documents, and they have to be indexed, I'm now using Jaguar Orm that uses SQLite as DB engine for this purpose, but the implementation is so so tricky and I want to use Sembast. So the question is: Do you have or will you have some mechanism to index the storage, or this is not necessary and it will have good performance in this scenario?

[alextekartik]

As sembast loads everything in memory, I would not advise using it to store 500 000 documents unless you have tons of memory. And no there is no index. If you do actually try it, I would be interested to know if it works and the performance! Sembast should be ok for small to medium database (whatever that means, I have tried up to 100 000 records of about 1kb each). For big databases, I'd rather user sqflite (without any ORM to optimize as much as possible)

[carotkut94]

what if I am storing 50,000 items and need to get the data in pages based on some category id, would it be fine with it? and if so then how do I implement the paging in dao class, and how do I replace item if two items have same id?

[alextekartik]

There is no dao support in sembast itself and I don't know how well it can play with existing solutions. I'm personnally not a fan of code generation that I don't have control on. Not sure about the kind of paging support you need. There is like in sqflite a notion of offset and limit, but it is not very efficient (if there is sorting, it still needs to sort everything first). Like on firestore, there is query mechanism you can listen to (store.query.onSnapshots) that should take care of properly updating the result for display without any paging needs.

Unfortunately, it is hard to give strong advices. I understand you don't want to get stuck with some unknown limitations but cannot at this point tell you whether it will match all your needs. As I said earlier, I use sqflite for big database. Once you try it, if you have some needs and suggestions I'll be all ears to suggestions and improvements....

[carotkut94]

ok, understood, but the real and basic question that i have to ask is, lets say i have a json { "id":1, "name":"Sidhant" }

and insert this into the "users" store now another user comes up with same id, but different name, i should ideally replace the old record with the same id, but it does no do that, it inserts it as a new record.

and I am trying to update the record but it always fails

Future<int> updateQuantity(CartModel cartModel) async {
    int updatedRows =
        await _categoryStore.update(await _db, {"quantity": cartModel.quantity},
            finder: Finder(
                filter: Filter.and([
              Filter.equals("product_id", cartModel.productId),
              Filter.equals("packageTypeId", cartModel.packageTypeId)
            ])));
    return updatedRows;
  }

and by paging i meant, lets say i have 100 products , and i dont want to load all of them at once in the listview, rather i will load 10 record in per page i.e i will be having the 10 pages having 10 records each and as soon as i scroll to end, i will load another page

[alextekartik]

Many questions at once, not all related.

1) you should make the id field the key of the record => userStore.record(map['id']).put(db, map) or use a transaction to find the existing record first if any before adding or upting it (userStore.findFirst(txn, finder: Finder(filter: Filter.equals('id', map['id'])))

2) Hard to says without seeing more code, the existing content, whether column names are corrent - some are camelCase some have under_score - , what you mean by not working...if you have more code (and event better a unit test) that explains the issue you are seeing can you fill a new issue as this is not related to the current one.

3) ok then offset/limit is your friend