search

Girbons / comics-downloader

tool to download comics and manga in pdf/epub/cbr/cbz from a website
MIT License
456 stars 49 forks source link

comicextra fails -all if there are some missing issues in a series #59

Closed jwhittaker closed 4 years ago

jwhittaker commented 4 years ago

Describe the bug -all flag crashes the program if there is a missing issue inbetween.

To Reproduce

./comic-dl -format cbz -all -url https://www.comicextra.com/comic/abc

Once it reaches a chapter page with no image files, the program completely exits with this error:

INFO[0000] Downloading...
[0s:0s]panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x6774f9]
goroutine 1 [running]:
image/jpeg.Encode(0x8e1040, 0xc0000a4050, 0x0, 0x0, 0x0, 0x0, 0x8e0b20)
        /usr/local/go/src/image/jpeg/writer.go:576 +0x29
github.com/Girbons/comics-downloader/pkg/util.SaveImage(0x8e1040, 0xc0000a4050, 0x7fe8f7db6130, 0xc00074c9c0, 0x8450ef, 0x3, 0x0, 0xb)
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/util/util.go:171 +0xcb
github.com/Girbons/comics-downloader/pkg/core.(*Comic).DownloadImages(0xc000142120, 0x7ffe672d1bed, 0x1, 0x0, 0x0, 0x0, 0x0)
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/core/core.go:262 +0x685
github.com/Girbons/comics-downloader/pkg/core.(*Comic).makeCBRZ(0xc000142120, 0x7ffe672d1bed, 0x1, 0x0, 0x0)        /Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/core/core.go:170 +0xab
github.com/Girbons/comics-downloader/pkg/core.(*Comic).MakeComic(0xc000142120, 0x7ffe672d1bed, 0x1, 0x100, 0x0)
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/core/core.go:289 +0x65
github.com/Girbons/comics-downloader/cmd/app.download(0xc0001a5f08, 0x0)
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/cmd/app/downloader.go:90 +0x65f
github.com/Girbons/comics-downloader/cmd/app.Run(0xc000049f08)
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/cmd/app/downloader.go:125 +0x89
main.main()
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/cmd/downloader/main.go:83 +0x245

Expected behavior The program would skip that chapter and move on to the next one until the list is exhausted. The program starts comicsextra at the latest chapter when -all is used, so it would finally stop after getting past #1.

Desktop (please complete the following information):

Additional context Attempts to work around this using bash

  1. The .cbz only contains the (first) cover image even if the issue is complete: 
    for xx in {1..240}; do ./comic-dl -format cbz -url https://www.comicextra.com/abc/chapter-$xx; done
  2. Produces a .pdf only with the (first) cover image. 
    for xx in {1..243}; do ./comic-dl -images-format jpg -url https://www.comicextra.com/abc/chapter-$xx; done
  3. Will only download the (first) cover image to issue dirs 
    for xx in {1..240}; do ./comic-dl -images-only -images-format jpg  -url https://www.comicextra.com/abc/chapter-$xx; done
Girbons commented 4 years ago

Hi @jwhittaker, thanks for reporting!

I'll take a look asap

Girbons commented 4 years ago

Hey @jwhittaker,

could you please give me an example url?

bashfulrobot commented 4 years ago

I just had this issues

   0% |                                        |  [0s:0s]panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x6774f9]

goroutine 1 [running]:
image/jpeg.Encode(0x8e1040, 0xc0000cc090, 0x0, 0x0, 0x0, 0x0, 0x8e0b20)
    /usr/local/go/src/image/jpeg/writer.go:576 +0x29
github.com/Girbons/comics-downloader/pkg/util.SaveImage(0x8e1040, 0xc0000cc090, 0x7f5379583e40, 0xc000326c90, 0x8450ef, 0x3, 0x0, 0x16)
    /Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/util/util.go:171 +0xcb
github.com/Girbons/comics-downloader/pkg/core.(*Comic).DownloadImages(0xc000178090, 0x844b84, 0x1, 0x0, 0x0, 0x0, 0x0)
    /Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/core/core.go:262 +0x685
github.com/Girbons/comics-downloader/pkg/core.(*Comic).makeCBRZ(0xc000178090, 0x844b84, 0x1, 0x0, 0x0)
    /Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/core/core.go:170 +0xab
github.com/Girbons/comics-downloader/pkg/core.(*Comic).MakeComic(0xc000178090, 0x844b84, 0x1, 0x8, 0x0)
    /Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/core/core.go:289 +0x65
github.com/Girbons/comics-downloader/cmd/app.download(0xc0001fbf08, 0xc00008de00)
    /Users/girbons/go/src/github.com/Girbons/comics-downloader/cmd/app/downloader.go:90 +0x65f
github.com/Girbons/comics-downloader/cmd/app.Run(0xc00008df08)
    /Users/girbons/go/src/github.com/Girbons/comics-downloader/cmd/app/downloader.go:125 +0x89
main.main()
    /Users/girbons/go/src/github.com/Girbons/comics-downloader/cmd/downloader/main.go:83 +0x245

Ran with comics-downloader -format cbz --all https://www.comicextra.com/captain-marvel-carol-danvers-the-ms-marvel-years

Girbons commented 4 years ago

Hey @bashfulrobot, thanks for reporting and giving an example url, I'll try to look into it soon

Girbons commented 4 years ago

@bashfulrobot it worked for me using the latest release, which version are you using?

bashfulrobot commented 4 years ago

I'll try updating and have another go. I have automation in place and thought I was the latest, but I'll manually confirm. Thank you.

bashfulrobot commented 4 years ago

OK,

Confirming version (just updated)

❯ comics-downloader --version
comics-downloader version v0.22.3

And the URL now works. Thank you for your help. I should have tried that first on my own.

Girbons commented 4 years ago

@bashfulrobot you're welcome!

ProgrammingJack commented 4 years ago

@Girbons I have the same error. With version v0.22.3.

$ ./comics-downloader -version
comics-downloader version v0.22.3
$ ./comics-downloader -format cbr -all https://www.comicextra.com/walt-disneys-comics-and-stories            
INFO[0000] Downloading...                                url="https://www.comicextra.com/walt-disneys-comics-and-stories"
   0% |                                        |  [0s:0s]panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x6774f9]

goroutine 1 [running]:
image/jpeg.Encode(0x8e1040, 0xc003f7c1e0, 0x0, 0x0, 0x0, 0x0, 0x8e0b20)
    /usr/local/go/src/image/jpeg/writer.go:576 +0x29
github.com/Girbons/comics-downloader/pkg/util.SaveImage(0x8e1040, 0xc003f7c1e0, 0x7fe036c160c8, 0xc003fdc420, 0x8450ef, 0x3, 0x0, 0xb)
    /Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/util/util.go:171 +0xcb
github.com/Girbons/comics-downloader/pkg/core.(*Comic).DownloadImages(0xc00013c090, 0x7ffe520990ac, 0x1, 0x0, 0x0, 0x0, 0x0)
    /Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/core/core.go:262 +0x685
github.com/Girbons/comics-downloader/pkg/core.(*Comic).makeCBRZ(0xc00013c090, 0x7ffe520990ac, 0x1, 0x0, 0x0)
    /Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/core/core.go:170 +0xab
github.com/Girbons/comics-downloader/pkg/core.(*Comic).MakeComic(0xc00013c090, 0x7ffe520990ac, 0x1, 0x200, 0x0)
    /Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/core/core.go:289 +0x65
github.com/Girbons/comics-downloader/cmd/app.download(0xc002b65f08, 0xc000043e00)
    /Users/girbons/go/src/github.com/Girbons/comics-downloader/cmd/app/downloader.go:90 +0x65f
github.com/Girbons/comics-downloader/cmd/app.Run(0xc000043f08)
    /Users/girbons/go/src/github.com/Girbons/comics-downloader/cmd/app/downloader.go:125 +0x89
main.main()
    /Users/girbons/go/src/github.com/Girbons/comics-downloader/cmd/downloader/main.go:83 +0x245

It starts downloading one image comics/www.comicextra.com/walt-disneys-comics-and-stories/images-chapter-743/0000-image.jpg, and then crashes. The file is completely empty (0 bytes).

--

edit: As additional info. the last issues also appear black on the website. So I guess it just can't download them, because the images behind the issues don't exist. However older issues do exist, e.g. https://www.comicextra.com/walt-disneys-comics-and-stories/chapter-738.

ProgrammingJack commented 4 years ago

Btw, a -issue option would be nice. Some series, like the one above ^, have way to many issues, and sometimes you only want one specific.

Girbons commented 4 years ago

Hi @ProgrammingJack,

Btw, a -issue option would be nice. Some series, like the one above ^, have way to many issues, and sometimes you only want one specific.

The tool already do that, you need to pass the issue url and omit the -all parameter.

It starts downloading one image comics/www.comicextra.com/walt-disneys-comics-and-stories/images-chapter-743/0000-image.jpg, and then crashes. The file is completely empty (0 bytes).

Thanks for reporting this issue I will investigate soon!

ProgrammingJack commented 4 years ago

The tool already do that, you need to pass the issue url and omit the -all parameter.

Then there must be also some other bug. ./comics-downloader -format cbr https://www.comicextra.com/walt-disneys-comics-and-stories/chapter-73 only downloads the first page, not the complete issue.

Girbons commented 4 years ago

@ProgrammingJack

Then there must be also some other bug. ./comics-downloader -format cbr https://www.comicextra.com/walt-disneys-comics-and-stories/chapter-73 only downloads the first page, not the complete issue.

The tool behavior is right in this case, because you are asking to download only one page.

Ask for the complete issue by specifying /full - https://www.comicextra.com/walt-disneys-comics-and-stories/chapter-73/full