29mb db file, memory jumps from 250mb to 1.5gb on db.open

[tobinharris]

We're using Sembast 3.1.1+1 with a 4,000 record database on sqflite, where the .db file is 29mb, Flutter mobile (iOS). After recently installing crash reporting, we're getting OOM errors and noticed that the app memory usage jumps from 250mb to 1.5gb when db.open() is called.

We're diagnosing this but wondered if you've seen it before?

[alextekartik]

Sembast is not memory friendly but the number looks huge. Great if you could share your database file.

After some experimentation, indeed, the database was loaded all at once which could cause some sqlite oom. I publish sembast_sqflite 2.0.1 that partially addresses that (and load them by 1000 records by default)

In case you encounter some sqlite out of memory error or if you want to speed up initial load you can modify the import page size (number of rows read at once).

// Change import page size (default is 1000)
// to use less memory
factory.sqfliteImportPageSize = 100;

// Change import page size (default is 1000)
// to speed up loading
factory.sqfliteImportPageSize = 100000;

[tobinharris]

Many thanks for looking at this so quickly Alex 🙌

Sadly, it didn't affect our memory footprint. Our memory usage is pretty constant after db.open().

We don't think the OOM is due to sqflite. Somehow our 29 mb file is being translated into a 1.5gb memory footprint (the app is 200mb memory without db.open() being called.

Is there anything we could be doing wrong? We have about 5 stores with keys as strings. Many are small but one has 4,000 records. The records aren't that big, but do have a few levels of nested JSON in them.

Any help is appreciated.