mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites
GNU General Public License v2.0
11.99k stars 977 forks source link

Connection closed/Forbidden error when downloading from TumblrGallery #6391

Closed com-network closed 3 weeks ago

com-network commented 3 weeks ago

I'm trying to download from TumblrGallery but I always get Connection closed/Forbidden errors. Opening the page in normal browser works just fine.

gallery-dl https://tumblrgallery.xyz/tumblrblog/gallery/370553.html -v -R 1 --print-traffic

[gallery-dl][debug] Version 1.27.7 - Executable (stable/windows)
[gallery-dl][debug] Python 3.8.10 - Windows-10-10.0.19045
[gallery-dl][debug] requests 2.32.3 - urllib3 2.2.3
[gallery-dl][debug] Configuration Files []
[gallery-dl][debug] Starting DownloadJob for 'https://tumblrgallery.xyz/tumblrblog/gallery/370553.html'
[tumblrgallery][debug] Using TumblrgalleryTumblrblogExtractor for 'https://tumblrgallery.xyz/tumblrblog/gallery/370553.html'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): tumblrgallery.xyz:443
send: b'GET /tumblrblog/gallery/370553.html HTTP/1.1\r\nHost: tumblrgallery.xyz\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:128.0) Gecko/20100101 Firefox/128.0\r\nAccept: */*\r\nAccept-Language: en-US,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nReferer: https://tumblrgallery.xyz/\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Tue, 29 Oct 2024 03:31:10 GMT
header: Server: Apache
header: Expires: Thu, 19 Nov 1981 08:52:00 GMT
header: Cache-Control: no-store, no-cache, must-revalidate
header: Pragma: no-cache
header: Set-Cookie: PHPSESSID=kr9e04gd5615e0dgj3fdis0rtg; path=/
header: Upgrade: h2
header: Connection: Upgrade, close
header: Vary: Accept-Encoding
header: Content-Encoding: gzip
header: Content-Length: 6535
header: Content-Type: text/html; charset=UTF-8
[urllib3.connectionpool][debug] https://tumblrgallery.xyz:443 "GET /tumblrblog/gallery/370553.html HTTP/11" 200 6535
[urllib3.connectionpool][debug] Resetting dropped connection: tumblrgallery.xyz
send: b'GET /tumblrblog/gallery/370553/1.html HTTP/1.1\r\nHost: tumblrgallery.xyz\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:128.0) Gecko/20100101 Firefox/128.0\r\nAccept: */*\r\nAccept-Language: en-US,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nReferer: https://tumblrgallery.xyz/\r\nCookie: PHPSESSID=kr9e04gd5615e0dgj3fdis0rtg\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Tue, 29 Oct 2024 03:31:11 GMT
header: Server: Apache
header: Expires: Thu, 19 Nov 1981 08:52:00 GMT
header: Cache-Control: no-store, no-cache, must-revalidate
header: Pragma: no-cache
header: Upgrade: h2
header: Connection: Upgrade, close
header: Vary: Accept-Encoding
header: Content-Encoding: gzip
header: Content-Length: 6527
header: Content-Type: text/html; charset=UTF-8
[urllib3.connectionpool][debug] https://tumblrgallery.xyz:443 "GET /tumblrblog/gallery/370553/1.html HTTP/11" 200 6527
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): 64.media.tumblr.com:443
send: b'GET /276221b8f263ca158cc33ed513cb0885/tumblr_pi269xQ7zV1s24tze_540.jpg HTTP/1.1\r\nHost: 64.media.tumblr.com\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:128.0) Gecko/20100101 Firefox/128.0\r\nAccept: */*\r\nAccept-Language: en-US,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nReferer: https://tumblrgallery.xyz/\r\n\r\n'
reply: 'HTTP/1.1 403 Forbidden\r\n'
header: Server: nginx
header: Date: Tue, 29 Oct 2024 03:31:11 GMT
header: Content-Type: text/html
header: Content-Length: 146
header: Connection: keep-alive
header: Access-Control-Allow-Methods: GET
header: Access-Control-Allow-Origin: *
header: Access-Control-Max-Age: 86400
header: Alt-Svc: h3=":443"; ma=86400
header: Strict-Transport-Security: max-age=31536000; preload
header: Server-Timing: dc;desc=hkg, cache;desc=BYPASS;dur=0.0
[urllib3.connectionpool][debug] https://64.media.tumblr.com:443 "GET /276221b8f263ca158cc33ed513cb0885/tumblr_pi269xQ7zV1s24tze_540.jpg HTTP/11" 403 146
[downloader.http][warning] '403 Forbidden' for 'https://64.media.tumblr.com/276221b8f263ca158cc33ed513cb0885/tumblr_pi269xQ7zV1s24tze_540.jpg'
[download][error] Failed to download tumblrgallery_370553_001_pi269xQ7zV1s24tze.jpg
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): 78.media.tumblr.com:443
send: b'GET /e670602ec486bf296d8d4d63f98b02c8/tumblr_msd5wjR0oY1rjjjkoo1_1280.png HTTP/1.1\r\nHost: 78.media.tumblr.com\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:128.0) Gecko/20100101 Firefox/128.0\r\nAccept: */*\r\nAccept-Language: en-US,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nReferer: https://tumblrgallery.xyz/\r\n\r\n'
reply: ''
[downloader.http][warning] ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) (1/2)
[urllib3.connectionpool][debug] Starting new HTTPS connection (2): 78.media.tumblr.com:443
send: b'GET /e670602ec486bf296d8d4d63f98b02c8/tumblr_msd5wjR0oY1rjjjkoo1_1280.png HTTP/1.1\r\nHost: 78.media.tumblr.com\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:128.0) Gecko/20100101 Firefox/128.0\r\nAccept: */*\r\nAccept-Language: en-US,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nReferer: https://tumblrgallery.xyz/\r\n\r\n'
reply: ''
[downloader.http][warning] ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) (2/2)
[download][error] Failed to download tumblrgallery_370553_002_msd5wjR0oY1rjjjkoo1.png
mikf commented 3 weeks ago

It is currently sending "https://tumblrgallery.xyz/" as Referer when downloading Tumblr media, which makes Tumblr ignore/block the request. You can disable this behavior with -o referer="".

$ gallery-dl https://tumblrgallery.xyz/tumblrblog/gallery/370553.html
[downloader.http][warning] '403 Forbidden' for 'https://64.media.tumblr.com/276221b8f263ca158cc33ed513cb0885/tumblr_pi269xQ7zV1s24tze_540.jpg'
[download][error] Failed to download tumblrgallery_370553_001_pi269xQ7zV1s24tze.jpg
...

$ gallery-dl -o referer="" https://tumblrgallery.xyz/tumblrblog/gallery/370553.html
./tumblrgallery/370553 idontspeakb…gallery_370553_001_pi269xQ7zV1s24tze.jpg
./tumblrgallery/370553 idontspeakb…llery_370553_002_msd5wjR0oY1rjjjkoo1.jpg
./tumblrgallery/370553 idontspeakb…llery_370553_003_msd5wjR0oY1rjjjkoo2.jpg
...