[pixiv] Ugoira postprocessor returns a TypeError

[komoreshi]

Ugoira artwork seem to be failing, resulting in partially finished (.part) files. Pixiv might've changed the way they're delivered?

gallery-dl https://www.pixiv.net/en/artworks/110530644 -v
gallery-dl: Version 1.27.0
gallery-dl: Python 3.12.3 - Windows-10-10.0.19045-SP0
gallery-dl: requests 2.31.0 - urllib3 2.1.0
gallery-dl: Configuration Files ['%APPDATA%\\gallery-dl\\config.json']
gallery-dl: Starting DownloadJob for 'https://www.pixiv.net/en/artworks/110530644'
pixiv: Using PixivWorkExtractor for 'https://www.pixiv.net/en/artworks/110530644'
urllib3.connectionpool: Starting new HTTPS connection (1): app-api.pixiv.net:443
urllib3.connectionpool: https://app-api.pixiv.net:443 "GET /v1/illust/detail?illust_id=110530644 HTTP/1.1" 200 651
postprocessor.ugoira: using concat demuxer
pixiv: Active postprocessor modules: [UgoiraPP]
urllib3.connectionpool: https://app-api.pixiv.net:443 "GET /v1/ugoira/metadata?illust_id=110530644 HTTP/1.1" 200 193
urllib3.connectionpool: Starting new HTTPS connection (1): i.pximg.net:443
urllib3.connectionpool: https://i.pximg.net:443 "GET /img-zip-ugoira/img/2023/08/04/23/40/37/110530644_ugoira1920x1080.zip HTTP/1.1" 416 190

postprocessor.ugoira: TypeError: expected string or bytes-like object, got 'list'
* .\gallery-dl\pixiv\21549100\21549100_110530644_p0.webm

Ugoira postprocessor if required:

{
        "name": "ugoira",
        "whitelist": ["pixiv", "danbooru"],
        "extension": "webm",
        "ffmpeg-args": ["-c:v", "libvpx-vp9", "-lossless", "1", "-pix_fmt", "yuv420p"],
        "ffmpeg-twopass": true,
        "ffmpeg-location": "ffmpeg"
},

[komoreshi]

Still can't pinpoint what's causing this. No ffmpeg output, so I imagine it's the completed file not being accepted by the ugoira postprocessor.

Different ffmpeg versions and gallery-dl version all seem to fail. No stack trace either, just postprocessor.ugoira: TypeError: expected string or bytes-like object, got 'list'

[mikf]

This is not an issue with FFmpeg, but something that happens in gallery-dl's code when generating FFmpeg commands and options.

I've tested your post processor settings from https://github.com/mikf/gallery-dl/issues/5683#issue-2330182828 and didn't get an error, so there's something else going on. Are you somehow modifying certain metadata data structure with a different post processor? Are you using other pixiv options that might be relevant here?

No stack trace either

I know ... Would have been really helpful if I had added them in 3346a377 Oh well, better late than never, right? https://github.com/mikf/gallery-dl/commit/4727ad6f572c1d003c95db4d36271227d63665d8

[komoreshi]

Weird, turned out to be the extractor.pixiv.actions portion of my config. Looks like the following part of my pixiv extractor config was responsible somehow as it works fine without it (it was only there temporarily until sanity_level was changed to an warning). Is it possible that was somehow being passed as an argument for ffmpeg?

"actions": {
    "debug:sanity_level": "level = error"
}

Here's the trace if you're interested (ty for the latest commit):

postprocessor.ugoira: TypeError: expected string or bytes-like object, got 'list'
postprocessor.ugoira:
Traceback (most recent call last):
  File "gallery_dl\postprocessor\ugoira.py", line 146, in convert
  File "gallery_dl\postprocessor\ugoira.py", line 175, in _exec
  File "gallery_dl\output.py", line 117, in log
TypeError: expected string or bytes-like object, got 'list'

Either way, happy to close this off or leave it open as a bug

[mikf]

Thank you for debugging this all on your own. Fixed in 9671bd6d40e5238709da8da52eb3b7c6d2bdfdaa.