search

mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites
GNU General Public License v2.0
11.7k stars 953 forks source link

[tumblr] an NSFW blog `https://datte-before-dawn.tumblr.com/` is viewable but fails to download #2023

Closed rautamiekka closed 2 years ago

rautamiekka commented 2 years ago

I figured I'll still mention it cuz the blog does show up in browser after clicking 'View this Tumblr' (which is weird since Tumblr had totally blocked viewing NSFW ones for years) but only in a sidebar.

Link=https://datte-before-dawn.tumblr.com/

My config=https://paste.gg/p/rautamiekka/4b59d554b54142e0bad7d1047b37d0b9

Output:

$ gallery-dl -v http://datte-before-dawn.tumblr.com/
gallery-dl: Version 1.19.2
gallery-dl: Python 3.9.7 - Windows-10-10.0.19043-SP0
gallery-dl: requests 2.26.0 - urllib3 1.26.7
gallery-dl: Starting DownloadJob for 'http://datte-before-dawn.tumblr.com/'
tumblr: Using custom api_key authentication
tumblr: Using TumblrUserExtractor for 'http://datte-before-dawn.tumblr.com/'
urllib3.connectionpool: Starting new HTTPS connection (1): api.tumblr.com:443
urllib3.connectionpool: https://api.tumblr.com:443 "GET /v2/blog/datte-before-dawn.tumblr.com/posts?offset=0&limit=50&reblog_info=true&api_key=REDACTED HTTP/1.1" 404 150

tumblr: NotFoundError: Requested user or post could not be found

$ gallery-dl -v https://www.tumblr.com/blog/view/datte-before-dawn
gallery-dl: Version 1.19.2
gallery-dl: Python 3.9.7 - Windows-10-10.0.19043-SP0
gallery-dl: requests 2.26.0 - urllib3 1.26.7
gallery-dl: Starting DownloadJob for 'https://www.tumblr.com/blog/view/datte-before-dawn'
gallery-dl: No suitable extractor found for 'https://www.tumblr.com/blog/view/datte-before-dawn'
Hrxn commented 2 years ago

First of all, you have to be sure that you are using the correct account credentials for API access, i.e. that key/token belong actually to an account that can access the blog.

This definitely used to work, but if I remember correctly, you have to provide the full set of OAuth tokens for your tumblr config part, i.e. like this:

            "access-token": "...",
            "access-token-secret": "...",
            "api-key": "...",
            "api-secret": "...",
rautamiekka commented 2 years ago

You might be mixing Tumblr with something cuz

  1. Tumblr never gives any access tokens (I just tried).
  2. https://github.com/mikf/gallery-dl/blob/master/docs/configuration.rst doesn't mention anything (tried searching for tumblr, access, and token).
  3. https://github.com/mikf/gallery-dl/blob/master/gallery_dl/extractor/tumblr.py doesn't mention anything (tried searching for token and access, and read the code).
  4. I can download another NSFW blog https://coatieyay.tumblr.com, one that's blocked from viewing (https://www.tumblr.com/blog/view/coatieyay works fine, though. Makes me suspect Tumblr has ulterior motives), just fine:
$ gallery-dl -v -d ./tumblr/ --download-archive "" https://coatieyay.tumblr.com
gallery-dl: Version 1.19.2
gallery-dl: Python 3.9.7 - Windows-10-10.0.19043-SP0
gallery-dl: requests 2.26.0 - urllib3 1.26.7
gallery-dl: Starting DownloadJob for 'https://coatieyay.tumblr.com'
tumblr: Using custom api_key authentication
tumblr: Using TumblrUserExtractor for 'https://coatieyay.tumblr.com'
urllib3.connectionpool: Starting new HTTPS connection (1): api.tumblr.com:443
urllib3.connectionpool: https://api.tumblr.com:443 "GET /v2/blog/coatieyay.tumblr.com/posts?offset=0&limit=50&reblog_info=true&api_key=REDACTED HTTP/1.1" 200 66253
tumblr: Active postprocessor modules: [MetadataPP, ComparePP]
urllib3.connectionpool: Starting new HTTPS connection (1): 64.media.tumblr.com:443
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /b52f59888a8c8eeab804d9e6a8339643/tumblr_inline_pjur1dXdOO1tvdsdv_1280.png HTTP/1.1" 200 1505
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_181181418279_01.png
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /49296376676ee9fbc91527c479baf453/tumblr_pjt32bRVzD1ux8kh1o1_1280.png HTTP/1.1" 200 210603
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_181153757914_01.jpg
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /2b33fc89e58fbe157095ebbdd0038c9e/tumblr_pjlak603oh1ux8kh1o1_1280.png HTTP/1.1" 200 547098
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_181025361489_01.png
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /d119829d6778f0e5e54b88b80e4a43ce/tumblr_pj8sxo2dMZ1ux8kh1o1_640.png HTTP/1.1" 200 116545
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_180812926794_01.png
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /a4de64c3cf02bfae41fbb8d51ea883ee/tumblr_pj6ikb6Twx1ux8kh1o1_1280.png HTTP/1.1" 200 734005
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_180765381464_01.png
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /68cf339bdfcd6b9d10b36156240786c3/tumblr_pivcwl5Tp11ux8kh1o1_1280.png HTTP/1.1" 200 169268
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_180561527199_01.jpg
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /da4b2ae7afce653ca50177ec6dd68b96/tumblr_piiojs3KMO1ux8kh1o1_1280.png HTTP/1.1" 200 752953
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_180325723629_01.png
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /9c0c73fd19d8aa249d1e40f3da9c3959/tumblr_piep2eDNzU1ux8kh1o1_1280.png HTTP/1.1" 200 672747
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_180249376544_01.png
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /0e283027d3d40372e9269972816a9cbe/tumblr_phzrjnN8Rn1ux8kh1o1_1280.png HTTP/1.1" 200 303195
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_179969560194_01.png
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /8b1a34bf545cc025fe81b10a76c1f278/tumblr_phsv6raKAb1ux8kh1o1_1280.png HTTP/1.1" 200 284615
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_179843063024_01.png
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /e226cb533530da4414cdd3c63f08eb94/tumblr_ph51aoKi9V1ux8kh1o1_1280.png HTTP/1.1" 200 546582
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_179407396419_01.png
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /27976799b276c8f9c312062275065226/tumblr_ph34sevjPY1ux8kh1o1_1280.png HTTP/1.1" 200 986609
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_179371864299_01.png
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /63328228d683b0a7bf4dcfbde3c1a3f4/tumblr_ph18prpur21ux8kh1o1_1280.png HTTP/1.1" 200 588908
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_179336166219_01.png
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /47191ea4a4f713c0d98d7f75ca84a19e/tumblr_ph0ehbmkTE1ux8kh1o1_1280.png HTTP/1.1" 200 781228
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_179319990179_01.png
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /da0eb3018d9666a0da538cf7be5e853b/tumblr_ph08rbUk8E1ux8kh1o1_r1_1280.png HTTP/1.1" 200 992533
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_179314574629_01.png
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /dcb11b67d77294f5fa5cc404e51b825f/tumblr_pgz5gk857z1ux8kh1o1_1280.png HTTP/1.1" 200 261513
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_179295610319_01.jpg
urllib3.connectionpool: https://api.tumblr.com:443 "GET /v2/blog/coatieyay.tumblr.com/posts?offset=50&limit=50&reblog_info=true&api_key=REDACTED HTTP/1.1" 200 31757
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /7e3139be99eb72ffd7ed963980fe9cf4/tumblr_pgtir1tPIH1ux8kh1o1_1280.png HTTP/1.1" 200 565160
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_179193802969_01.png
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /253f1f9fa488d7906f4f0f84177bbb2e/tumblr_pgpsbvVcL41ux8kh1o1_1280.png HTTP/1.1" 200 305612
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_179126845424_01.png
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /1d6f36c8543008aed579c84c64d198b5/tumblr_pgb36pW7wr1ux8kh1o1_1280.png HTTP/1.1" 200 440122
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_178870447119_01.png
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /b8f421bfc00177e70d546e341378230c/tumblr_pfzktyG2lU1ux8kh1o1_1280.png HTTP/1.1" 200 766639
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_178673828979_01.png
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /612b395355865daa850e7afed7b7caca/tumblr_pfsd41FM0V1ux8kh1o1_1280.png HTTP/1.1" 200 1016962
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_178552679384_01.png
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /d45c3a51d8cc97f2dfab453df19f624c/tumblr_pf6edfjeCA1ux8kh1o1_1280.png HTTP/1.1" 200 203877
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_178163129784_01.png
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /a279bb842c5fed88c3e12e88b3bc8a76/tumblr_pecbvsg4iX1ux8kh1o1_1280.png HTTP/1.1" 200 227446
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_177596091354_01.jpg
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /5f5b4b1363e31dfafb2bf9e0af26efe1/tumblr_pddhlj0Tvr1ux8kh1o1_1280.png HTTP/1.1" 200 332654
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_176931780189_01.jpg
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /0bfe1c0cac53ad7165d13f657ada5de2/tumblr_pcos7b7c2I1ux8kh1o1_1280.jpg HTTP/1.1" 200 175074
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_176447612579_01.jpg
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /ee3a3f53809a5b8f7fb9fb5abf060881/tumblr_pcos7b7c2I1ux8kh1o2_1280.jpg HTTP/1.1" 200 85556
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_176447612579_02.jpg
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /05587c8277a08898eb458a4c397ee8bf/tumblr_pcos7b7c2I1ux8kh1o4_1280.jpg HTTP/1.1" 200 208453
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_176447612579_03.jpg
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /293c04226b74643edb772acea0e4389b/tumblr_pcos7b7c2I1ux8kh1o3_1280.jpg HTTP/1.1" 200 251597
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_176447612579_04.jpg
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /4b3bcf4f37f735617f8ff8aac2cd8798/tumblr_pcos7b7c2I1ux8kh1o5_1280.jpg HTTP/1.1" 200 197037
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_176447612579_05.jpg
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /6508f8af76f9fd5d0e8edad1b621d14f/tumblr_pb5gb1GaCp1ux8kh1o1_1280.png HTTP/1.1" 200 220088
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_175411546619_01.jpg
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /d9c4f3325bea56e88d504c87b1319ae1/tumblr_paleraGCZx1ux8kh1o1_1280.png HTTP/1.1" 200 997952
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_175279377171_01.png
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /0fc535c413a57936669462a778b4770b/tumblr_palf0iietD1ux8kh1o1_r1_1280.png HTTP/1.1" 200 229660
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_175179045609_01.jpg
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /59aa363ed8fec4b4191b382c036221c5/tumblr_p9vt1cEJKj1ux8kh1o1_1280.png HTTP/1.1" 200 787462
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_174636916690_01.png
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /11d973fcf90c45a529891a87de784aab/tumblr_p8u8y0c1Eq1ux8kh1o1_1280.png HTTP/1.1" 200 271066
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_173967099204_01.jpg
urllib3.connectionpool: https://64.media.tumblr.com:443 "GET /402de5709b30b5669b8efb6d8d82ef90/tumblr_p7yq89vDeR1ux8kh1o1_1280.png HTTP/1.1" 200 1073147
* .\tumblr\tumblr\coatieyay\tumblr_coatieyay_173425214704_01.png
urllib3.connectionpool: https://api.tumblr.com:443 "GET /v2/blog/coatieyay.tumblr.com/posts?offset=100&limit=50&reblog_info=true&api_key=REDACTED HTTP/1.1" 200 43416

KeyboardInterrupt
mikf commented 2 years ago

No, he's right, you need an access-token and access-token-secret to be able to access that first blog. You can get them with gallery-dl oauth:tumblr. That makes 4 API/OAuth tokens when also using a custom api-key and api-secret, which is the same for all sites using OAuth1.0. (flickr, smugmug, and tumblr)

rautamiekka commented 2 years ago

Huh, didn't even occur to me that might be needed. I stand corrected. Glad I reported it, no telling how long it would had taken otherwise.