mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites
GNU General Public License v2.0
11.87k stars 976 forks source link

`FilenameFormatError` on field value that is `None` when using conversion option #4423

Closed Hrxn closed 1 year ago

Hrxn commented 1 year ago

Just found this..

PS C:\Apps\Temp> python.exe C:\Apps\z_repo_public\gallery-dl-master\gallery_dl\__main__.py 'https://www.redgifs.com/watch/basicmonthlyleonberger' --config-ignore --option filename="{num:?/./>04}{gallery:?/./[:11]}{date:%Y-%m-%d}.{description!t:?/./R.//}{filename}.{userName}.{extension}"
[redgifs][error] FilenameFormatError: Applying filename format string failed (TypeError: descriptor 'strip' for 'str' objects doesn't apply to a 'NoneType' object)
NativeCommandExitException: Program "python.exe" ended with non-zero exit code: 32.
PS C:\Apps\Temp> python.exe C:\Apps\z_repo_public\gallery-dl-master\gallery_dl\__main__.py 'https://www.redgifs.com/watch/basicmonthlyleonberger' --config-ignore --option filename="{num:?/./>04}{gallery:?/./[:11]}{date:%Y-%m-%d}.{description!t:?/./R.//}{filename}.{userName}.{extension}" --verbose
[gallery-dl][debug] Version 1.26.0-dev
[gallery-dl][debug] Python 3.11.4 - Windows-10-10.0.19045-SP0
[gallery-dl][debug] requests 2.31.0 - urllib3 2.0.4
[gallery-dl][debug] Configuration Files []
[gallery-dl][debug] Starting DownloadJob for 'https://www.redgifs.com/watch/basicmonthlyleonberger'
[redgifs][debug] Using RedgifsImageExtractor for 'https://www.redgifs.com/watch/basicmonthlyleonberger'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): api.redgifs.com:443
[urllib3.connectionpool][debug] https://api.redgifs.com:443 "GET /v2/auth/temporary HTTP/1.1" 200 None
[urllib3.connectionpool][debug] https://api.redgifs.com:443 "GET /v2/gifs/basicmonthlyleonberger HTTP/1.1" 200 None
[redgifs][error] FilenameFormatError: Applying filename format string failed (TypeError: descriptor 'strip' for 'str' objects doesn't apply to a 'NoneType' object)
NativeCommandExitException: Program "python.exe" ended with non-zero exit code: 32.

Proof that it's working if I remove the !t conversion specifier:

PS C:\Apps\Temp> python.exe C:\Apps\z_repo_public\gallery-dl-master\gallery_dl\__main__.py 'https://www.redgifs.com/watch/basicmonthlyleonberger' --config-ignore --option filename="{num:?/./>04}{gallery:?/./[:11]}{date:%Y-%m-%d}.{description:?/./R.//}{filename}.{userName}.{extension}" --verbose
[gallery-dl][debug] Version 1.26.0-dev
[gallery-dl][debug] Python 3.11.4 - Windows-10-10.0.19045-SP0
[gallery-dl][debug] requests 2.31.0 - urllib3 2.0.4
[gallery-dl][debug] Configuration Files []
[gallery-dl][debug] Starting DownloadJob for 'https://www.redgifs.com/watch/basicmonthlyleonberger'
[redgifs][debug] Using RedgifsImageExtractor for 'https://www.redgifs.com/watch/basicmonthlyleonberger'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): api.redgifs.com:443
[urllib3.connectionpool][debug] https://api.redgifs.com:443 "GET /v2/auth/temporary HTTP/1.1" 200 None
[urllib3.connectionpool][debug] https://api.redgifs.com:443 "GET /v2/gifs/basicmonthlyleonberger HTTP/1.1" 200 None
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): thumbs46.redgifs.com:443
[urllib3.connectionpool][debug] https://thumbs46.redgifs.com:443 "GET /BasicMonthlyLeonberger.mp4?expires=1692168600&signature=v2:438059cc0129ded1be19c733498f58c3ba459c1561640e4ae015a7520bf1c6a1&for=2a00:6020:b314:8e00&hash=6163438793 HTTP/1.1" 404 None
[downloader.http][warning] '404 Not Found' for 'https://thumbs46.redgifs.com/BasicMonthlyLeonberger.mp4?expires=1692168600&signature=v2:438059cc0129ded1be19c733498f58c3ba459c1561640e4ae015a7520bf1c6a1&for=2a00:6020:b314:8e00&hash=6163438793'
[download][info] Trying fallback URL #1
[urllib3.connectionpool][debug] Resetting dropped connection: thumbs46.redgifs.com
[urllib3.connectionpool][debug] https://thumbs46.redgifs.com:443 "GET /BasicMonthlyLeonberger-mobile.mp4?expires=1692168600&signature=v2:807167489fae945b7e420d29a054eaf9632eb0951bd2d5fbfcb5cbfc874822b6&for=2a00:6020:b314:8e00&hash=6163438793 HTTP/1.1" 200 1287809
* .\gallery-dl\redgifs\2021-06-09.BasicMonthlyLeonberger.chuck_bronson.mp4
PS C:\Apps\Temp>

Well I guess it is basically expected in this case, because this example on redgifs (a SFW one basically.. I mean something you can also see on Instagram, I'd say) does not have a description, but some clips there now have..

But maybe it shouldn't fail even without a value for description?

mikf commented 1 year ago

It has been like this, conversions raising an exception when applied to anything but the expected type, since forever.

You should just use {description|''!t} to make sure t gets a string.

Hrxn commented 1 year ago

I just realized that this has been the setting in my config for "filename", and I've downloaded a bunch of gigs these days and it seemed to be working just fine?

"{num:?/./>04}{gallery:?/./[:11]}{date:%Y-%m-%d}.{tags|'Untagged':?/./J_/}{description[:80]!t:?/./R.//}{filename}.{width}x{height}.{extension}"

This means I haven't reliably reproduced it, apparently..

Anyway, with the alternative you suggested:

"{num:?/./>04}{gallery:?/./[:11]}{date:%Y-%m-%d}.{tags|'Untagged':?/./J_/}{description[:80]|''!t:?/./R.//}{filename}.{width}x{height}.{extension}"

I think I'm gonna close this