NookLibrary 0.1.3 runs out of memory with library of about 3500-4000 books

[GoogleCodeExporter]

While running logcat on adb, I see that NookLibrary runs out of memory.  I have 
about 3500 ebooks on my external SDHC card, and about 500 ebooks purchased from 
B&N, and about half of the B&N ebooks are archived.  It usually runs out of 
memory before it finishes scanning all the books.

Most of the ebooks on the external SDHC card have covers, so I suspect that if 
NookLibrary is caching cover thumbnails in memory, and running out of memory. I 
suspect the average size thumbnail is about 4k, and 4,000 * 4k takes up the 
entire 16MB allowed for the application.

Original issue reported on code.google.com by bgalbre...@gmail.com on 23 Oct 2010 at 4:15

[GoogleCodeExporter]

no. images are not cached. But keywords and descriptions are. I think we can 
fix the memory error by clearing old data.
can you copy the stacktrace in log and attach it here?
I probably won't be able to test it since I don't have that many books. I'll 
attach the apk file here so that others can test and report results.

Original comment by hari.swa...@gmail.com on 25 Oct 2010 at 1:57

• Changed state: Accepted

[GoogleCodeExporter]

I have the same problem. I tried to load about 3,100 books and it runs out of 
memory. It will be great to fix this, since this is the main reason to root the 
nook, and there is no sense in having a 8GB sd card if you can't use this with 
you sideload books with coverflow.

Hari, how do you get the stakrace in log in your nook so i can attach it here?

Thanks,

Original comment by stefanol...@gmail.com on 2 Dec 2010 at 3:35

[GoogleCodeExporter]

'adb logtrace' should give the stacktrace & other recent logs. Just run this in 
a window & then start the app.

Original comment by hari.swa...@gmail.com on 2 Dec 2010 at 3:56

[GoogleCodeExporter]

Hari, I ran the logcat with about 3,200 books...

Attached you can find the log.

Hope you can fix it.

Thanks!

Original comment by stefanol...@gmail.com on 3 Dec 2010 at 12:59

Attachments:

• logcat.txt

[GoogleCodeExporter]

Thanks. I have been busy working with the new nookColor sdk this week. I'll 
look into this next week. 

Original comment by hari.swa...@gmail.com on 5 Dec 2010 at 5:48

[GoogleCodeExporter]

Thanks, let me know if you need to get a different logcat, or need me to test 
something... As i say, for me this is the best functionality of rooting...

Let me know.

Original comment by stefanol...@gmail.com on 5 Dec 2010 at 7:03

[GoogleCodeExporter]

The library memory problem seems to be very sensitive to size of descriptions.  
My limit went from over 3000 to less than 2500 when I removed 300 short stories 
and added 100 novels.  I set up Calibre to report the length of the 
descriptions and found one book with a 29K byte field. (Someone had put the 
entire first chapter in the description)  Cleaning up that one and 10 over 2K 
(Calibre will keep adding to this field if you pull metadata more than once) 
brought it back up to 3100.

Original comment by jplo...@gmail.com on 5 Dec 2010 at 10:00

[GoogleCodeExporter]

Any tips on how to do the report of the length of description on Calibre?

Original comment by stefanol...@gmail.com on 5 Dec 2010 at 10:17

[GoogleCodeExporter]

I used the create catalog CSV option with just author, title, and comments 
selected.
Then opened it with openoffice calc and used the LEN() function against the 
comments field. It seemed like the quickest way for a one off report.

Original comment by jplo...@gmail.com on 5 Dec 2010 at 11:48

[GoogleCodeExporter]

please test with the attached apk and let me know if it fixes the problem. If 
it crashes, collect the logcat output and attach to this incident like before.

Thanks.

Original comment by hari.swa...@gmail.com on 9 Dec 2010 at 10:59

Attachments:

• nookLibrary.apk

[GoogleCodeExporter]

Thanks Hari... I'll try it later and let you know

Original comment by stefanol...@gmail.com on 9 Dec 2010 at 11:16

[GoogleCodeExporter]

Hari... It didn't work.

Attached the resultant logcat

Original comment by stefanol...@gmail.com on 10 Dec 2010 at 4:47

Attachments:

• logcat.txt

[GoogleCodeExporter]

Thanks. I'll debug further over the weekend. 

Original comment by hari.swa...@gmail.com on 10 Dec 2010 at 5:07

[GoogleCodeExporter]

Try with this one. I made some additional changes to not cache data. I'll 
probably have to rewrite the logic to handle this case. But don't want to slow 
down users with less books.

Thanks,
Hari

Original comment by hari.swa...@gmail.com on 15 Dec 2010 at 10:20

Attachments:

• nookLibrary.apk

[GoogleCodeExporter]

Thanks, i'll try it tonight.

Original comment by stefanol...@gmail.com on 15 Dec 2010 at 11:49

[GoogleCodeExporter]

It fails.

It doesn't seems to read my sideload library... it only reads B&N library...

Attached logcat.txt

Original comment by stefanol...@gmail.com on 16 Dec 2010 at 1:35

Attachments:

• logcat.txt

[GoogleCodeExporter]

ok. there must be something else going wrong. 
It loads all the books for me with ~900 books. I guess I have to get more books 
and test it myself. Thanks for your help. 
I'll copy more books and test it.

Original comment by hari.swa...@gmail.com on 16 Dec 2010 at 1:57

[GoogleCodeExporter]

Hari, i tried installing it again and reseting my nook.

It goes trying to load all books but when it finish indexing crashes as the 
other versions...

Original comment by stefanol...@gmail.com on 16 Dec 2010 at 12:51

[GoogleCodeExporter]

can you get the log when it crashes? 

Original comment by hari.swa...@gmail.com on 16 Dec 2010 at 6:09

[GoogleCodeExporter]

Sure... I don't know if it can be done, or you have already done that, but 
maybe you can compile nooklibrary.apk with more messages to give more info on 
logtrace...

Anyway at night i'll give you input on this with this version...

Original comment by stefanol...@gmail.com on 16 Dec 2010 at 6:15

[GoogleCodeExporter]

It already logs the line number in source code whenever an exception occurs.
But, since out of memory error is a generic case, fix usually involves reducing 
the total memory used in the app ( by reading from database instead of keeping 
in memory etc). 
if the exception is due to something else, logcat output should have the stack 
trace and the exact line numbers.

Original comment by hari.swa...@gmail.com on 16 Dec 2010 at 7:24

[GoogleCodeExporter]

Hari... attached the logcat for the last version of nooklibrary...

Original comment by stefanol...@gmail.com on 17 Dec 2010 at 2:37

Attachments:

• logcat.txt

[GoogleCodeExporter]

Hi,
the log stacktrace shows the line numbers for old code.
Can you check the version nbr in the top? does it say 0.1.4?
You should also have 2 new menu options just above Help- screen saver and show 
progress.  

Original comment by hari.swa...@gmail.com on 17 Dec 2010 at 3:28

[GoogleCodeExporter]

Hari,

The version number is the old one : 0.1.3...

What i'm doing is first uninstalling with adb uninstall com.nookdevs.library 
and then install nooklibrary.apk with the last version attached here.

Could you verify if the last attached here is 0.1.4? 

Original comment by stefanol...@gmail.com on 17 Dec 2010 at 3:59

[GoogleCodeExporter]

sorry. Looks like I added the old apk. Here is the latest one.

Original comment by hari.swa...@gmail.com on 17 Dec 2010 at 4:39

Attachments:

• nookLibrary.apk

[GoogleCodeExporter]

Just installed it... tomorrow i'll load 1,000 more books to test it...

What i've noticed on this version is :

1. It indexes the books VERY fast
2. no cover appears... all appears with the gray "No cover available" cover...

Original comment by stefanol...@gmail.com on 17 Dec 2010 at 5:01

[GoogleCodeExporter]

Hari, it worked...

I attached the logcat with some sort options i made on library...

Thanks!

Original comment by stefanol...@gmail.com on 17 Dec 2010 at 1:24

Attachments:

• logcat.txt

[GoogleCodeExporter]

Thanks! I'll remove some of the extra logs and upload the apk. 

Original comment by hari.swa...@gmail.com on 17 Dec 2010 at 1:29

[GoogleCodeExporter]

Just 2 comments, now that i'm reviewing this version :

1. It's just keep the message of refresh in progress forever...but it indexes...
2. Some of the new books doesn't show covers (No cover available)

Beisdes that everything works perfectly

Original comment by stefanol...@gmail.com on 17 Dec 2010 at 2:48

[GoogleCodeExporter]

Maybe it just need more time... I'll try another logcat at night...

Original comment by stefanol...@gmail.com on 17 Dec 2010 at 5:33

[GoogleCodeExporter]

I tried 1.4 last night.  The load behavior looks to have changed as it seems to 
be more of a background task after the initial sort, but not all of the 
functions work until the refreshing message goes away. (e.g. covers, show, 
authors). I like this as it is usable while refreshing.  I did notice that 
using these particular functions slows down the ongoing indexing.  I put the 
nook aside after starting a refresh and waited until the refreshing message 
went away and all of the functions, including the covers worked.

Original comment by jplo...@gmail.com on 17 Dec 2010 at 6:13

[GoogleCodeExporter]

Here is the latest apk. I fixed few more bugs in this one.

Original comment by hari.swa...@gmail.com on 17 Dec 2010 at 9:56

Attachments:

• nookLibrary.apk

[GoogleCodeExporter]

Thanks! Will try it later and let you know...

Original comment by stefanol...@gmail.com on 17 Dec 2010 at 10:20

[GoogleCodeExporter]

Hari,

Could it be that there is some sub-process running with this version, that 
drains my battery?

When i installed it my battery goes down very quickly, i uninstalled it and 
install 0.1.3 and everything ok....

Original comment by stefanol...@gmail.com on 20 Dec 2010 at 12:58

[GoogleCodeExporter]

I didn't notice any difference. I'll run it tonight and check in the morning.
There shouldn't be any difference between the previous version and this one 
because after refresh is complete, everything should be idle.

Original comment by hari.swa...@gmail.com on 20 Dec 2010 at 2:11

[GoogleCodeExporter]

Ok... I have tried this version for some days, but it seems to have some 
trouble with the refresh or sorting options.... It just keep refreshing or 
sorting without stop (hence consuming battery more than the other version). 
Sometimes it finish refreshing, other times it seems to keep on processing...

I seem to have this problems even with less than 2000 books...I'm returning now 
to the last version, which is a pity, because this one adds many interesting 
features...

It seems to needs a little more debugging...

Original comment by stefanol...@gmail.com on 24 Dec 2010 at 4:39

[GoogleCodeExporter]

I tried with 4000 epubs and it took an hr to finish refresh and didn't see any 
other issues. I'll do more test after the holidays. 
If you see any errors/stack traces in the log, copy it here and ill use that to 
debug.

Thanks,
Hari

Original comment by hari.swa...@gmail.com on 24 Dec 2010 at 1:33

[GoogleCodeExporter]

Maybe i'm desperate with the refreshing times and i only need to wait. 

Original comment by stefanol...@gmail.com on 24 Dec 2010 at 2:50

[GoogleCodeExporter]

You should release it on Nookdevs apps, so many more people can test on it...

Original comment by stefanol...@gmail.com on 24 Dec 2010 at 2:52

[GoogleCodeExporter]

Hi, Hari.

I was wondering if you will you be releasing this version soon to nookdevs?

Original comment by stefanol...@gmail.com on 30 Dec 2010 at 11:04

[GoogleCodeExporter]

How long should it take for it to load information from sideloaded content? I 
installed the version from comment 32, and while it displayed the books, it 
never loaded the metadata from my 4k books, even when I had it running My Books 
and let it run for several hours.  I just deleted everything, and installed 
about 100 books, and none of the metadata has loaded for the 100 books after 
about 10 minutes.  The metadata from the B&N content is showing up, but I 
presume you're getting that from the B&N catalog DB.

Original comment by bgalbre...@gmail.com on 31 Dec 2010 at 9:45

[GoogleCodeExporter]

It took a bit over an hr for indexing 4000 epubs. 
100 books should finish in a couple of mins or so. 
Could be something else. You probably have to build the code & debug using 
eclipse to see what's going on.

Original comment by hari.swa...@gmail.com on 31 Dec 2010 at 10:06