search

mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites
GNU General Public License v2.0
11.7k stars 953 forks source link

Unexpected error while downloading from Pixiv, instructed by command prompt to report here #6285

Closed baconmeh closed 1 week ago

baconmeh commented 2 weeks ago 
C:\Users\Me II\Desktop\essentials>gallery-dl https://www.pixiv.net/en/users/101904824/artworks/%E3%83%A9%E3%83%93%E3%83%83%E3%83%88%E3%83%9B%E3%83%BC%E3%83%AB --verbose
[gallery-dl][debug] Version 1.27.5 - Executable (stable/windows)
[gallery-dl][debug] Python 3.8.10 - Windows-10-10.0.17763
[gallery-dl][debug] requests 2.31.0 - urllib3 2.1.0
[gallery-dl][debug] Configuration Files ['%APPDATA%\\gallery-dl\\config.json']
[gallery-dl][debug] Starting DownloadJob for 'https://www.pixiv.net/en/users/101904824/artworks/%E3%83%A9%E3%83%93%E3%83%83%E3%83%88%E3%83%9B%E3%83%BC%E3%83%AB'
[pixiv][debug] Using PixivArtworksExtractor for 'https://www.pixiv.net/en/users/101904824/artworks/%E3%83%A9%E3%83%93%E3%83%83%E3%83%88%E3%83%9B%E3%83%BC%E3%83%AB'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): app-api.pixiv.net:443
[urllib3.connectionpool][debug] https://app-api.pixiv.net:443 "GET /v1/user/illusts?user_id=101904824 HTTP/1.1" 200 4819
[postprocessor.ugoira][debug] using concat demuxer
[pixiv][debug] Active postprocessor modules: [UgoiraPP]
[urllib3.connectionpool][debug] https://app-api.pixiv.net:443 "GET /v1/ugoira/metadata?illust_id=122823412 HTTP/1.1" 200 519
.\gallery-dl\pixiv\101904824 mikenekopunch23\122823412_p0.webm
[urllib3.connectionpool][debug] https://app-api.pixiv.net:443 "GET /v1/ugoira/metadata?illust_id=122167314 HTTP/1.1" 200 517
.\gallery-dl\pixiv\101904824 mikenekopunch23\122167314_p0.webm
[urllib3.connectionpool][debug] https://app-api.pixiv.net:443 "GET /v1/ugoira/metadata?illust_id=121710966 HTTP/1.1" 200 519
.\gallery-dl\pixiv\101904824 mikenekopunch23\121710966_p0.webm
[urllib3.connectionpool][debug] https://app-api.pixiv.net:443 "GET /v1/ugoira/metadata?illust_id=121307872 HTTP/1.1" 200 481
.\gallery-dl\pixiv\101904824 mikenekopunch23\121307872_p0.webm
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): i.pximg.net:443
[urllib3.connectionpool][debug] https://i.pximg.net:443 "GET /img-original/img/2024/08/08/00/04/30/121277908_p0.png HTTP/1.1" 416 190
[pixiv][error] An unexpected error occurred: BadZipFile - File is not a zip file. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[pixiv][debug]
Traceback (most recent call last):
  File "gallery_dl\job.pyc", line 152, in run
  File "gallery_dl\job.pyc", line 196, in dispatch
  File "gallery_dl\job.pyc", line 361, in handle_url
  File "gallery_dl\postprocessor\ugoira.pyc", line 145, in convert_from_zip
  File "zipfile.pyc", line 1269, in __init__
  File "zipfile.pyc", line 1336, in _RealGetContents
zipfile.BadZipFile: File is not a zip file

Occurred while processing the command gallery-dl https://www.pixiv.net/en/users/101904824/artworks/%E3%83%A9%E3%83%93%E3%83%83%E3%83%88%E3%83%9B%E3%83%BC%E3%83%AB, only being stopped by https://www.pixiv.net/en/artworks/121277908.

Strangely, I can just input gallery-dl https://www.pixiv.net/en/artworks/121277908 and it'll download fine. As easy a fix as that is, I imagine this report should help prevent this from happening in the future.

baconmeh commented 2 weeks ago

Update, occurred again. This has been occurring multiple times while trying to download this gallery, and each time I either just run the command again or select the individual post ID that's bugging out.


[gallery-dl][debug] Version 1.27.5 - Executable (stable/windows)
[gallery-dl][debug] Python 3.8.10 - Windows-10-10.0.17763
[gallery-dl][debug] requests 2.31.0 - urllib3 2.1.0
[gallery-dl][debug] Configuration Files ['%APPDATA%\\gallery-dl\\config.json']
[gallery-dl][debug] Starting DownloadJob for 'https://www.pixiv.net/en/users/88356170'
[pixiv][debug] Using PixivUserExtractor for 'https://www.pixiv.net/en/users/88356170'
[pixiv][debug] Using PixivArtworksExtractor for 'https://www.pixiv.net/users/88356170/artworks'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): app-api.pixiv.net:443
[urllib3.connectionpool][debug] https://app-api.pixiv.net:443 "GET /v1/user/illusts?user_id=88356170 HTTP/1.1" 200 8261
[postprocessor.ugoira][debug] using concat demuxer
[pixiv][debug] Active postprocessor modules: [UgoiraPP]
[urllib3.connectionpool][debug] https://app-api.pixiv.net:443 "GET /v1/ugoira/metadata?illust_id=119229422 HTTP/1.1" 200 366
.\gallery-dl\pixiv\88356170 yuzuhara1017\118723439_p0.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\118723439_p1.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\118632092_p0.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\118400948_p0.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\118400948_p1.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\118356890_p0.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\118356890_p1.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\118356890_p2.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\118356890_p3.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\118354491_p0.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\118101072_p0.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\118101072_p1.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\118101072_p2.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\118101072_p3.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\118009825_p0.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\118009825_p1.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\117904161_p0.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\117829495_p0.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\117829495_p1.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\117580483_p0.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\117574270_p0.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\117556130_p0.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\117534453_p0.jpg
[urllib3.connectionpool][debug] https://app-api.pixiv.net:443 "GET /v1/user/illusts?user_id=88356170&offset=30 HTTP/1.1" 200 12039
[urllib3.connectionpool][debug] https://app-api.pixiv.net:443 "GET /v1/ugoira/metadata?illust_id=117507303 HTTP/1.1" 200 365
.\gallery-dl\pixiv\88356170 yuzuhara1017\117507303_p0.webm
[urllib3.connectionpool][debug] https://app-api.pixiv.net:443 "GET /v1/ugoira/metadata?illust_id=117439004 HTTP/1.1" 200 455
.\gallery-dl\pixiv\88356170 yuzuhara1017\117439004_p0.webm
.\gallery-dl\pixiv\88356170 yuzuhara1017\117388736_p0.png
.\gallery-dl\pixiv\88356170 yuzuhara1017\117353062_p0.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\117334270_p0.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\117210044_p0.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\116885726_p0.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\116808188_p0.png
.\gallery-dl\pixiv\88356170 yuzuhara1017\116553190_p0.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\116553190_p1.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\116553190_p2.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\116553190_p3.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\116553190_p4.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\116553190_p5.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\116553190_p6.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\116553190_p7.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\116553190_p8.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\116498397_p0.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\116498397_p1.jpg
[urllib3.connectionpool][debug] https://app-api.pixiv.net:443 "GET /v1/ugoira/metadata?illust_id=116482407 HTTP/1.1" 200 372
.\gallery-dl\pixiv\88356170 yuzuhara1017\116482407_p0.webm
.\gallery-dl\pixiv\88356170 yuzuhara1017\116352432_p0.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\116352432_p1.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\116352432_p2.jpg
.\gallery-dl\pixiv\88356170 yuzuhara1017\116220121_p0.jpg
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): i.pximg.net:443
[urllib3.connectionpool][debug] https://i.pximg.net:443 "GET /img-original/img/2024/02/13/16/30/55/116015725_p0.jpg HTTP/1.1" 200 None
.\gallery-dl\pixiv\88356170 yuzuhara1017\116015725_p0.jpg[pixiv][error] An unexpected error occurred: BadZipFile - File is not a zip file. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[pixiv][debug]
Traceback (most recent call last):
  File "gallery_dl\job.pyc", line 152, in run
  File "gallery_dl\job.pyc", line 196, in dispatch
  File "gallery_dl\job.pyc", line 361, in handle_url
  File "gallery_dl\postprocessor\ugoira.pyc", line 145, in convert_from_zip
  File "zipfile.pyc", line 1269, in __init__
  File "zipfile.pyc", line 1336, in _RealGetContents
zipfile.BadZipFile: File is not a zip file
moulins commented 2 weeks ago

I've hit what I think is the same bug; from what I can see, this only triggers when dowloading non-ugoiras after a ugoira was skipped due to an archive check.

I did some code spelunking, and it seems the bug happens because UgoiraPP.prepare is called for the skipped ugoira, which sets self._convert_zip = True.
Normally, this member is set to False again when the processing happens in UgoiraPP.convert_from_zip, but because the ugoira is skipped this is never called, and the preprocessor will mistakenly consider the next file it encounters as an ugoira to be decompressed.