search

Cortys / comic-backup

Back up your comics as CBZ.
https://cmxlgy.wordpress.com
GNU General Public License v3.0
295 stars 33 forks source link

Keeps skipping pages and crashing. #64

Closed tjw01 closed 6 years ago

tjw01 commented 7 years ago

The backup script is doing my head in at the moment. As of a couple of weeks ago it was working perfectly, but in the last week it's been skipping a huge number of pages (irrespective of what I set the swap interval and skip delay to), and if it isn't skipping pages it's just crashing completely mid-way with the "Aw Snap" page.

I've tried forcing a new exploit scan, completely reinstalling as well as a fresh install on a totally new computer but nothing I've tried has solved it.

Cortys commented 7 years ago

This is probably related to #51. My current assumption is, that this is due to Chromes new CPU throttling feature for background tabs. I'm currently working on a fix.

For now you could try opening the reader tab during backups, this should prevent the throttling.

tjw01 commented 7 years ago

Ah, I thought it might be a Chrome update or something. Glad to hear you're working on it!

boltagon commented 7 years ago

I seem to have a similar problem. Worked fine last week now either crashes with "snap" error message, or produces a .cbz file with only a partial copy of the comic/collection. The size of the book doesn't seem to matter and I still get errors with the read tab open. I have tried re-installing the extension (v 2.3.3) and forcing a new exploit scan with no luck. Any tips / patches would be appreciated. Thanks.

Cortys commented 7 years ago

The fix should be out within the next couple of days.

For now, as I mentioned, try keeping the backup tab in the foreground. This should help.

AndrewD2 commented 7 years ago

Not sure if it's related. Yesterday when I went to do some backups of very large comics (~300 pages) I would get an out of memory error around page 180. This was with the tab in the foreground.

boltagon commented 7 years ago

The problems occur with the tab in the forground too.

boltagon commented 7 years ago

doesn't seem to make a differenc whether the tab is foreground or background.

On 4/5/17, Andrew Betts notifications@github.com wrote:

Not sure if it's related. Yesterday when I went to do some backups of very large comics (~300 pages) I would get an out of memory error around page

  1. This was with the tab in the foreground.

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/Cortys/comic-backup/issues/64#issuecomment-292025732

Cortys commented 7 years ago

Interesting. I can reproduce the page skipping issues, but were not able to produce the crashes that are being reported.

Does this happen with every comic you tested, or only some of them?

boltagon commented 7 years ago

The crashes tend to be only on longer comics, for example "Mighty Avengers" Complete Collection, 474 pages. It does not crash every time, sometimes it just skips pages. I have managed to download one 33 page comic with no problems today but that is from about 30 attempts. I have tried adjusting the paging delays in the extension options, but no joy.

Cortys commented 7 years ago

Did the 33 page comic also crash or just skip pages?

boltagon commented 7 years ago

That only skipped pages

Cortys commented 7 years ago

The current master branch should fix the skipping issues that were reported and possibly also reduce the amount of page crashes (though I could not verify this yet).

Can you guys test the fix and tell me if it helped?

bzfoster commented 7 years ago

I grabbed the most recent and tried to scan some larger ones around 160ish pages. With the latest, it Aw Snapped for me whether I had focus or not. Before it would skip pages if I didn't have focus but would save the file and Aw Snap if I had focus. Still works with some 30ish page comics. I brought up the chrome task manager and the tab that was scanning hit about 2GB memory usage at around 60% finished on the 160ish page comic and was at 3GB around 90%. Let me know if there are any logs or anything else I get you to help.

sgbeal commented 7 years ago

Just tried with the latest master: on all 3 books i tried (2 long, 1 normal) it's hanging on the first page and (curiously) shows the "this tab is playing audio" icon (but no audio is playing). i re-ran the setup twice, but that didn't change the behaviour. i have increased both of the time-based config options to 3 seconds, but no change.

Version 57.0.2987.133 (64-bit) on Linux Mint 18.1

boltagon commented 7 years ago

Just loaded the latest master and it does seem to solve the page skipping problem. Still crashes with either an "aw Snap!" or an "out of memory" chrome error if I try anything larger than about 100 pages. At the moment I am working around this by:

  1. Changing the output container to be "jpeg" so I can easily track where it is in the backup.
  2. Alter the option so it backs up from from current page.
  3. Restarting the scan after a crash.
  4. Zipping the directory and renaming it manually.

A little more work, but it gets me backups of my comics so I can live with it.

Hopefully you will be able to come up with something better. Thanks for all your hard work.

Cortys commented 7 years ago

Just a quick status update. I found that debugging the crashing issue is more difficult than expected.

I could only reproduce the crashes on Linux, every other platform works fine for me. Does anybody here also have problems on MacOS or Windows?

Currently I'm not quite sure if the issue is easily fixable since it appears to be caused by the GC running too rarely and thus causing the tabs heap to overflow. But I might be wrong about that (the Chrome debuggers heap dumps appear to be incomplete)... I will get back to you, once I know more.

bzfoster commented 7 years ago

I'm experiencing it in Windows.

tobiasj42 commented 7 years ago

I am also having this issue with any book over 100 pages. It will consistently crash with an Aw Snap error with it in the foreground. I can actually watch it crash going from one page to the next. I am on Windows 10 with a Quad core I7 , 16GB of ram, and a SSD. Smaller books seem to be working correctly though I have not checked them for missing pages. I have also disabled background tab throttling (chrome://flags/#expensive-background-timer-throttling) which made no difference. This is a recent change, as a book the same size I was able to successfully scan about a month ago also crashes now.

tobiasj42 commented 7 years ago

Is there any update on this request?

boltagon commented 7 years ago

crashing problem still evident.

On 5/6/17, tobiasj42 notifications@github.com wrote:

Is there any update on this request?

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/Cortys/comic-backup/issues/64#issuecomment-299601244

sgbeal commented 7 years ago

i still cannot get the backup to advance past page 1 of any book. ALL books, regardless of their size, hang at 0% on the first page. While trying again today i noticed...

When the backer-upper is running, the tab shows the "this tab is playing audio" icon. When browsing the reader manually (without the backer-upper) it does NOT show that icon. This suggests that maybe the backup is doing something it shouldn't be doing (but i cannot speculate as to what that might be).

boltagon commented 7 years ago

the problem seems to have cleared for me. i downloaded 4 books over the last week varying from 300 -700 pages and they all scanned cleanly in one pass. not sure why this has suddenly cleared up now but will keep you updated.

On 5/17/17, Stephan Beal notifications@github.com wrote:

i still cannot get the backup to advance past page 1 of any book. ALL books, regardless of their size, hang at 0% on the first page. While trying again today i noticed...

When the backer-upper is running, the tab shows the "this tab is playing audio" icon. When browsing the reader manually (without the backer-upper) it does NOT show that icon. This suggests that maybe the backup is doing something it shouldn't be doing (but i cannot speculate as to what that might be).

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/Cortys/comic-backup/issues/64#issuecomment-302199144

sgbeal commented 7 years ago

Weird - i just tried one of the same books which kept hanging at 0% and it now also works as well.

sgbeal commented 7 years ago

Correction/addition: it scanned to completion but didn't name the download sensibly. It left a file with a UUID/hash as its name, with no file extension. It seems to be a complete CBZ, but has to be renamed manually. Not a major problem, but curious.

TurinTur commented 7 years ago

I had this problem but I 'solved' it selecting the reader tab, as suggested.

Cortys commented 6 years ago

@sgbeal Since you have been using the current master (with the fixes from last year) for quite a while now, how predictable do you think the extension currently behaves (regarding missing pages and crashes)?

sgbeal commented 6 years ago

TL;DR: it's as reliable as can possibly be expected.

i have backed up well over 1600 books using this tool and never once, ever, using any version, seen it skip a page. i don't always double-check them immediately, but (A) i sometimes check them right after scanning and (B) i almost always read scanned CBZs on my tablet rather than downloading them again via the Comixology app, so 95% or more of my scanned books get read within a couple weeks of scanning them.

It's conceivably possible that something i scanned but haven't yet read (or scanned after i read it in the Comixology app) is missing a page or two, but i've never seen it happen in the many thousands of pages i have scanned and read.

Edit: i've also never seen it crash. EDIT again: i remember having Aw Snap issues at one point but they were almost certainly caused by a particular version (or group of versions) of Chrome/Chromium, as they were (unless i'm sorely mistaken) eventually resolved without any update of this app.

Also, almost every time i use this app, i first run git pull and (re)install it to make sure i have the latest code.

(i have, once or twice, asked myself "is a page missing here?" because a story seems to skip something, but in those cases i have always verified that no pages were missing.)

Cortys commented 6 years ago

Great, I'll try to finally wrap up the 2.4.0 release then (even though I'm assuming most people are already using it anyways by having switched to the master branch).

boltagon commented 6 years ago

i have had a couple of instances in the last month or so where the extension has been slow at saving the files after zippg. i traced t to my virus scan app and now turn this off when doing backups. generally it has been vety stable.

On 8/29/18, Clemens Damke notifications@github.com wrote:

Great, I'll try to finally wrap up the 2.4.0 release then (even though I'm assuming most people are already using it anyways by having switched to the master branch).

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/Cortys/comic-backup/issues/64#issuecomment-416997507

Cortys commented 6 years ago

Thanks for the reports!

2.4.0 is now finally released. It includes a fix for this issue, so I'll close this now.