search

tekartik / sembast.dart

Simple io database
BSD 2-Clause "Simplified" License
786 stars 64 forks source link

Any performance metrics? #14

Closed jagan999 closed 4 years ago

jagan999 commented 5 years ago

I'm seriously considering usage of a NoSQL database for my Flutter app. I'm curious if there has been any performance benchmarking etc of this database, since it says the entire database is loaded into memory.

alextekartik commented 5 years ago

I have my own benchmark https://github.com/tekartik/sembast.dart/blob/master/sembast/doc/perf_io.md which are not really relevant, mainly to make sure any changes won't affect it.

Otherwise it depends on the usage. Big databases will increase initial load time and complex queries could impact reading but I don't have strong numbers yet. But yes since it is in memory it should be fast enough for regular usage in flutter even with 10K+ records.

jagan999 commented 5 years ago

Thanks for the express reply. I am anticipating data in the 100,000 to million. Do you suspect a perf issue?

alextekartik commented 5 years ago

Ouch. The issue with million is the size here as it might not fit in memory. Here sqlite would be a better fit especially since you can select fields when reading.

markmooibroek commented 5 years ago

I see

I have my own benchmark https://github.com/tekartik/sembast.dart/blob/master/doc/perf_io.md which are not really relevant, mainly to make sure any changes won't affect it.

Otherwise it depends on the usage. Big databases will increase initial load time and complex queries could impact reading but I don't have strong numbers yet. But yes since it is in memory it should be fast enough for regular usage in flutter even with 10K+ records.

Link is broken :-) https://github.com/tekartik/sembast.dart/blob/master/sembast/doc/perf_io.md

alextekartik commented 5 years ago

Thanks for rectifying! like you said the proper link is https://github.com/tekartik/sembast.dart/blob/master/sembast/doc/perf_io.md

taliptako commented 5 years ago

@alextekartik are those put or read ?

alextekartik commented 5 years ago

It is put performance. However these numbers are not really useful since writes happen lazily. I'm open to suggestion or PR on some actual performance metrics. Everything is memory but some algorythm could be improved.

bitsydarel commented 5 years ago

Do you plan to allow users to choose between loading the full database in memory or keeping it in disk?

Currently you're the only flutter nosql solution, production ready, but the fact that it's loading the full database in memory seems bad for mobile.

alextekartik commented 5 years ago

Unfortunately that is not the current design. I agree that loading in memory is pretty bad (I even found that very stupid initially). I personnally use sqflite on mobile for big databases. Although everybody hates sql, it remains the best solution.

I started sembast initially just to have a small pure dart database system I could use for desktop testing and for mocking other databases (indexed_db). flutter was not a target.

Alternative solution would be to wrap other no sql native solutions (realm, leveldb, couch) but I don't have any need/project/time for it but would be happy to work on other solutions.