search

marcucio / Cordova-WP-SqlitePlugin

14 stars 20 forks source link

Out of memory exception #25

Open gabrielmuresan opened 10 years ago

gabrielmuresan commented 10 years ago

I tested the plugin and it seems that this plugin can't handle 60 000 rows in ~8 tables. It gives out of memory exception while handling the operations.

The routine is like this.

  1. startup -> we insert in a table (ServerLogs) the information received from server ~60 000 rows now (which can be bigger or smaller)
  2. -> the information in the serverlogs is then processed in batches and then the necessary changes to the other tables is made in this order: Insert id 1 in BarTable Delete id 1 from Serverlogs Insert id 2 in FooTable Delete id 2 from ServerLogs ... and so on

Looking at the application memory usage it just grows almost constantly after inserting the 60 000 rows the memory usage is somewhere near 190mb and while moving them from serverlogs to the other tables it grows even faster letting me to only be able to handle 10 000 - 15 000 rows in batches of 1000. if the batch is increased, the out of memory error comes earlier in the process.

The maximum memory on a 1GB Windows Phone 8 is 300 mb.

Do you know where it might be the problem in the plugin? did you encounter memory leaks or any memory issues? DO you have some suggestions?

marcucio commented 10 years ago

I'm not sure where the memory leak is coming from, it could be from your JS or from the plugin.

By "batches or 1000" do you mean 1000 queries in each transition? If you only put 1 query in each transaction do you see a memory leak? This can help us narrow down where the leak is coming from.

gabrielmuresan commented 10 years ago

it's just one query "SELECT * FROM ServerLogs LIMIT 1000" for example, and running it 60 times until all ServerLog rows are gone. (because I am deleting them after they are applied)

gabrielmuresan commented 10 years ago

everything I do the memory keeps filling up. do you have any idea where this could come from?

marcucio commented 10 years ago

I'm not sure, I'm not seeing it in my apps. I'm not going to have much time to look into this issue so if you have the ability to look into it then it would be helpful.

gabrielmuresan commented 10 years ago

if the problem would come from the plugin. do you have any idea where it could be from? what part of the plugin would you suggest on starting to check for this error? just to narrow my search a bit.

thank you in advance

marcucio commented 10 years ago

Not sure, I haven't done much to the library in about a year.

It could be coming from the Javascript, maybe the callbacks we save are not getting deleted?

Or it could be coming from the c-sharp sqlite library itself where if this is the case then I don’t really know much about the inner workings of the library.

Mike Marcucio mike@marcucio.com

Website: http://marcucio.com Blog: http://getitdoneapp.com/blog Twitter: https://twitter.com/GetItDoneBlog Facebook: https://www.facebook.com/GetItDoneTasks Google+: https://plus.google.com/+Marcucio

On Apr 14, 2014, at 2:54 AM, gabrielmuresan notifications@github.com wrote:

if the problem would come from the plugin. do you have any idea where it could be from? what part of the plugin would you suggest on starting to check for this error? just to narrow my search a bit.

thank you in advance

— Reply to this email directly or view it on GitHub.