Out Of Memory

[GoogleCodeExporter]

What steps will reproduce the problem?
1.Not Sure
2.
3.

What is the expected output? What do you see instead?

What version of the product are you using? On what operating system?
r171

Please provide any additional information below.
Okay so this Ruleset Collection (My DC Collection , is much smaller than my 
full set (in which the DC Group is currently unfinished) but if i run this 
collection on my DC books it *says* runs out of memory before even finishing 
the first book but calculating from my task manager readings I still have 2G of 
memory free, however I do notice that both my CPUs are maxed out while Data 
Manager is running (this happens with any decent sized rule collection as well)

I'm thinking that the problem may lie with the fact that i have comicrack 
limited to only be allowed to use 512MB of RAM. On average it never goes above 
256MB (unless i turn off the limitations then the memory usage can shoot up 
past 1.5GB). Running Data Manager brings it up to it's 512 Limit which is when 
i promptly get this error. Is there possible anyway to streamline the memory 
usage for Data Manager?

Original issue reported on code.google.com by NiceziaVincent on 18 May 2013 at 4:00

Attachments:

• dataman.dat
• Image1.png

[GoogleCodeExporter]

will check this.

Original comment by rainer.c...@googlemail.com on 18 May 2013 at 5:31

• Changed state: Accepted

[GoogleCodeExporter]

on how many books (total) should this script had been running when the error 
occured? Have you tried it with a higher amount of CR memory (> 512 MB)?

Original comment by rainer.c...@googlemail.com on 18 May 2013 at 5:34

[GoogleCodeExporter]

as you can see in the bottom left hand corner i was only running it on 20 
books, the error happened on the first book (after it had already found the 
rule for the book and applied it, and yes it runs fine if i don't limit my 
memory usage but it also eats up all available memory when running Data Manager.

Original comment by NiceziaVincent on 18 May 2013 at 5:38

[GoogleCodeExporter]

perhaps it would be better to run one rule at a time over all books, rather 
than running all the rules on each individual book. I don't know if this will 
make a difference, but i'm almost certain that keeping all those dynamically 
compiled rules in memory is part of what causes this, and could be a big 
problem for people who still run on PCs with 1gig or less memory.

Original comment by NiceziaVincent on 18 May 2013 at 5:43

[GoogleCodeExporter]

Here's a picture of my CPU and Memory monitoring

Yellow is prior to Data Manager running, Red Is while Data manager Is Running, 
and Blue is once it hits the error. 

For Reference I'm running on a Dual Core 2.4Ghz (there are two processors 
monitored in the image. with 4GB of memory (3.5GB Availble to the system 512MB 
dedicated to video) 

on the left you can see that my memory usage drops to 2.27GB during running 
Data Manager it spikes to 3.4GB. My processor usage is 19 percent after Data 
manager stops, you can see in the image just before it stops my second core 
processor usage hits the very ceiling of my processor's abilities.

Original comment by NiceziaVincent on 18 May 2013 at 9:35

Attachments:

• Image2.png

[GoogleCodeExporter]

I agree, this looks critical. Maybe this results from the plugin architecture 
(I've had similar high CPU/RAM peaks with Library Organizer and CV Scraper from 
time to time). But I will check of course. Still need some time as at the 
moment my comic libraries are moving to their new destinations and I have no 
chance to test it right now. 

Original comment by rainer.c...@googlemail.com on 18 May 2013 at 9:53

[GoogleCodeExporter]

what happens if you give CR a bit more memory?

Original comment by rainer.c...@googlemail.com on 18 May 2013 at 10:11

[GoogleCodeExporter]

this is run with all limitations turned off...

I'm thinking the error may have something to do with some specific rule (not 
sure which it could be if so) because I'm starting to hit this error even with  
no limitations on ComicRack (just takes longer to hit it with the limitations 
off) above that's a 15 minute span of time that Data manager was running and it 
made it through 3 books.

Original comment by NiceziaVincent on 18 May 2013 at 10:47

[GoogleCodeExporter]

I will check tomorrow the code that is generated from your rules. Another thing 
to test might be what happens when you turn the automatic update of files off.

Original comment by rainer.c...@googlemail.com on 18 May 2013 at 10:56

[GoogleCodeExporter]

it is off, as i never use it.

Original comment by NiceziaVincent on 18 May 2013 at 10:59

[GoogleCodeExporter]

But I understood that you write your data to the files (with comicinfo.xml).

Original comment by rainer.c...@googlemail.com on 18 May 2013 at 11:02

[GoogleCodeExporter]

I manually do that (right click and "update book file"), I've had it happen in 
the past that automatic updates have rendered my book files unreadable so i 
don't chance it with automatic updates anymore

Original comment by NiceziaVincent on 18 May 2013 at 11:13

[GoogleCodeExporter]

good news... after closing and reopening comicrack the memory error disapeared, 
however the High CPU problem remains.

I got suspicious once every plugin started throwing an error after a few 
minutes of running... so i restarted ComicRack and poof no more errors

Original comment by NiceziaVincent on 19 May 2013 at 3:29

[GoogleCodeExporter]

should be fixed with new version of parser

Original comment by rainer.c...@googlemail.com on 5 Jun 2013 at 3:28

• Changed state: Started

[GoogleCodeExporter]

fixed with 1.2.0 (no more during runtime generated code)

Original comment by rainer.c...@googlemail.com on 10 Jun 2013 at 9:58

• Changed state: Fixed