mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites
GNU General Public License v2.0
11.7k stars 953 forks source link

[pixiv] artwork not downloading #6339

Closed mmoisall closed 2 days ago

mmoisall commented 1 week ago

it supposed to download user's all artwork but missing one of them

gallery-dl https://www.pixiv.net/users/56514424
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-09-06 112630 - samoinu - pixiv - 雨-0(84199360_p0).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-09-01 135800 - samoinu - pixiv - 9月-0(84093524_p0).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-08-27 124309 - samoinu - pixiv - 日周-0(83972265_p0).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-07-29 153509 - samoinu - pixiv - 夏-0(83314977_p0).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-07-24 025538 - samoinu - pixiv - おはよう。-0(83182542_p0).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-07-22 030945 - samoinu - pixiv - ⭐️-0(83137722_p0).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-07-21 115255 - samoinu - pixiv - 背景-0(83124871_p0).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-07-21 115255 - samoinu - pixiv - 背景-1(83124871_p1).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-07-21 115255 - samoinu - pixiv - 背景-2(83124871_p2).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-07-21 115255 - samoinu - pixiv - 背景-3(83124871_p3).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-07-21 115255 - samoinu - pixiv - 背景-4(83124871_p4).png

but it's working when I use the artwork's link

gallery-dl https://www.pixiv.net/artworks/85960783
[pixiv][warning] 85960783: 'sanity_level' warning
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-0(85960783_p0).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-1(85960783_p1).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-2(85960783_p2).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-3(85960783_p3).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-4(85960783_p4).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-5(85960783_p5).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-6(85960783_p6).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-7(85960783_p7).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-8(85960783_p8).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-9(85960783_p9).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-10(85960783_p10).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-11(85960783_p11).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-13(85960783_p13).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-14(85960783_p14).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-15(85960783_p15).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-16(85960783_p16).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-17(85960783_p17).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-18(85960783_p18).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-19(85960783_p19).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-20(85960783_p20).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-21(85960783_p21).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-22(85960783_p22).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-23(85960783_p23).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-24(85960783_p24).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-25(85960783_p25).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-26(85960783_p26).png
.\gallery-dl\pixiv - 56514424 - samoinu -\2020-11-28 103300 - samoinu - pixiv - 2020-1-27(85960783_p27).png

my config

    "pixiv": {
      "directory": [
        "pixiv - {user[id]} - {user[name]} -"
      ],
      "filename": "{date:%Y-%m-%d %H%M%S} - {user[name]} - pixiv - {title}-{num}({filename}).{extension}",
      "refresh-token": "*****",
      "sanity-level": 6,
      "ugoira": true,
    },

version : gallery_dl 1.27.6

what should I do to download all images with user id?

mikf commented 6 days ago

Duplicate of #4327

mikf commented 6 days ago

You've already opened another issue in this regard: #5435

/artwork/ URLs work now, but downloading restricted works by user ID has not been resolved yet.

espressoelf commented 3 days ago

/artwork/ URLs work now, but downloading restricted works by user ID has not been resolved yet.

Does this apply to downloads initiated from a bookmarks-list, too? I'm trying to download my favorites, https://www.pixiv.net/en/users/0000000/bookmarks/artworks?rest=hide, and there's one favorite that makes gallery-dl crash. I'm running the dev version, pulled 2024-10-19 20:26:24.

[urllib3.connectionpool][debug] https://www.pixiv.net:443 "GET /ajax/illust/104582860 HTTP/11" 404 37
[pixiv][error] An unexpected error occurred: TypeError - 'NoneType' object is not subscriptable (key 'urls'). Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[pixiv][debug]
Traceback (most recent call last):
  File "/opt/pypy/lib/pypy3.10/site-packages/gallery_dl/job.py", line 151, in run
    for msg in extractor:
  File "/opt/pypy/lib/pypy3.10/site-packages/gallery_dl/extractor/pixiv.py", line 65, in items
    files = self._extract_files(work)
  File "/opt/pypy/lib/pypy3.10/site-packages/gallery_dl/extractor/pixiv.py", line 122, in _extract_files
    return self._extract_ajax(work, body)
  File "/opt/pypy/lib/pypy3.10/site-packages/gallery_dl/extractor/pixiv.py", line 189, in _extract_ajax
    url = self._extract_ajax_url(body)
  File "/opt/pypy/lib/pypy3.10/site-packages/gallery_dl/extractor/pixiv.py", line 243, in _extract_ajax_url
    original = body["urls"]["original"]
TypeError: 'NoneType' object is not subscriptable (key 'urls')

The ajax call returns

{"error":true,"message":"","body":[]}

I hope this isn't the wrong issue, there's quite a lot of them open currently.

mikf commented 2 days ago

Detecting limit_sanity_level works for /users/ID/artworks results is now possible: https://github.com/mikf/gallery-dl/commit/75674944f0f030faf0cb61ee2d49a957570edbce. It needs a PHPSESSID cookie to be able to detect R-18 works, though.

@espressoelf Should be fixed in https://github.com/mikf/gallery-dl/commit/2d15e64ea9846ce20643201fe4afb34e4bc0e763. It previously caught only some but not all exceptions when handling empty results.