search

mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites
GNU General Public License v2.0
11.7k stars 953 forks source link

Patreon Extractor Gets 403 Forbidden #342

Closed Aster-the-Med-Stu closed 5 years ago

Aster-the-Med-Stu commented 5 years ago

Verbose output:

root@comic:~/OneDrive/Downloads# gallery-dl https://www.patreon.com/mick39 --write-metadata --write-tags --write-log log.log --cookies ./cookies.txt -v
[gallery-dl][debug] Version 1.8.7
[gallery-dl][debug] Python 3.6.8 - Linux-4.15.0-54-generic-x86_64-with-Ubuntu-18.04-bionic
[gallery-dl][debug] requests 2.22.0 - urllib3 1.25.3
[gallery-dl][debug] Starting DownloadJob for 'https://www.patreon.com/mick39'
[gallery-dl][debug] updating default urllib3 ciphers
[patreon][debug] Using PatreonCreatorExtractor for 'https://www.patreon.com/mick39'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): www.patreon.com:443
[urllib3.connectionpool][debug] https://www.patreon.com:443 "GET /mick39 HTTP/1.1" 302 None
[urllib3.connectionpool][debug] https://www.patreon.com:443 "GET /mick39/posts HTTP/1.1" 403 None
[patreon][error] HTTP request failed:  403: Forbidden for url: https://www.patreon.com/mick39

I believe that this is not a platform-specific error as I could reproduce this on my laptop with Windows 10 and Anaconda3 installed.

Besides, the Patreon extractor is also unusable for free contents(without session_id set):

  base  ~\..\Downloads  gallery-dl https://www.patreon.com/twokinds --write-metadata --write-tags --write-log log.log -v      [gallery-dl][debug] Version 1.8.7
[gallery-dl][debug] Python 3.7.3 - Windows-10-10.0.18362-SP0
[gallery-dl][debug] requests 2.22.0 - urllib3 1.24.2
[gallery-dl][debug] Starting DownloadJob for 'https://www.patreon.com/twokinds'
[gallery-dl][debug] updating default urllib3 ciphers
[patreon][debug] Using PatreonCreatorExtractor for 'https://www.patreon.com/twokinds'
[patreon][warning] no 'session_id' cookie set
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): www.patreon.com:443
[urllib3.connectionpool][debug] https://www.patreon.com:443 "GET /twokinds HTTP/1.1" 200 None
[urllib3.connectionpool][debug] https://www.patreon.com:443 "GET /api/posts?include=user,attachments,user_defined_tags,campaign,poll.choices,poll.current_user_responses.user,poll.current_user_responses.choice,poll.current_user_responses.poll,access_rules.tier.null&fields[post]=change_visibility_at,comment_count,content,current_user_can_delete,current_user_can_view,current_user_has_liked,embed,image,is_paid,like_count,min_cents_pledged_to_view,post_file,published_at,patron_count,patreon_url,post_type,pledge_url,thumbnail_url,teaser_text,title,upgrade_url,url,was_posted_by_campaign_owner&fields[user]=image_url,full_name,url&fields[campaign]=avatar_photo_url,earnings_visibility,is_nsfw,is_monthly,name,url&fields[access_rule]=access_rule_type,amount_cents&sort=-published_at&filter[is_draft]=false&filter[contains_exclusive_posts]=true&filter[campaign_id]=145535&json-api-use-default-includes=false&json-api-version=1.0 HTTP/1.1" 403 None
[patreon][error] HTTP request failed:  403: Forbidden for url: https://www.patreon.com/api/posts?include=user,attachments,user_defined_tags,campaign,poll.choices,poll.current_user_responses.user,poll.current_user_responses.choice,poll.current_user_responses.poll,access_rules.tier.null&fields[post]=change_visibility_at,comment_count,content,current_user_can_delete,current_user_can_view,current_user_has_liked,embed,image,is_paid,like_count,min_cents_pledged_to_view,post_file,published_at,patron_count,patreon_url,post_type,pledge_url,thumbnail_url,teaser_text,title,upgrade_url,url,was_posted_by_campaign_owner&fields[user]=image_url,full_name,url&fields[campaign]=avatar_photo_url,earnings_visibility,is_nsfw,is_monthly,name,url&fields[access_rule]=access_rule_type,amount_cents&sort=-published_at&filter[is_draft]=false&filter[contains_exclusive_posts]=true&filter[campaign_id]=145535&json-api-use-default-includes=false&json-api-version=1.0
ghost commented 5 years ago

had you tried to install pyOpenSSL? pip3 pyopenssl It might solve your problem.

Aster-the-Med-Stu commented 5 years ago

@ucok66 Er... Both on my Ubuntu server and Anaconda in Windows have pyopenssl installed.

And could you kindly test downloading something from Patreon on your machine?

ghost commented 5 years ago 
[ucok66@ucok66 ~]$ gallery-dl -v https://www.patreon.com/vytz
[gallery-dl][debug] Version 1.9.0-dev
[gallery-dl][debug] Python 3.7.3 - Linux-5.2.1-1-MANJARO-x86_64-with-arch-Manjaro-Linux
[gallery-dl][debug] requests 2.22.0 - urllib3 1.25.3
[gallery-dl][debug] Starting DownloadJob for 'https://www.patreon.com/vytz'
[patreon][debug] Using PatreonCreatorExtractor for 'https://www.patreon.com/vytz'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): www.patreon.com:443
[urllib3.connectionpool][debug] https://www.patreon.com:443 "GET /vytz HTTP/1.1" 302 None
[urllib3.connectionpool][debug] https://www.patreon.com:443 "GET /vytz/posts HTTP/1.1" 200 None
[urllib3.connectionpool][debug] https://www.patreon.com:443 "GET /api/posts?include=user,attachments,user_defined_tags,campaign,poll.choices,poll.current_user_responses.user,poll.current_user_responses.choice,poll.current_user_responses.poll,access_rules.tier.null&fields%5Bpost%5D=change_visibility_at,comment_count,content,current_user_can_delete,current_user_can_view,current_user_has_liked,embed,image,is_paid,like_count,min_cents_pledged_to_view,post_file,published_at,patron_count,patreon_url,post_type,pledge_url,thumbnail_url,teaser_text,title,upgrade_url,url,was_posted_by_campaign_owner&fields%5Buser%5D=image_url,full_name,url&fields%5Bcampaign%5D=avatar_photo_url,earnings_visibility,is_nsfw,is_monthly,name,url&fields%5Baccess_rule%5D=access_rule_type,amount_cents&sort=-published_at&filter%5Bis_draft%5D=false&filter%5Bcontains_exclusive_posts%5D=true&filter%5Bcampaign_id%5D=247331&json-api-use-default-includes=false&json-api-version=1.0 HTTP/1.1" 200 None
[urllib3.connectionpool][debug] https://www.patreon.com:443 "GET /api/user/141898 HTTP/1.1" 200 None
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): c10.patreonusercontent.com:443
[urllib3.connectionpool][debug] https://c10.patreonusercontent.com:443 "GET /3/e30%3D/patreon-media/p/post/28365302/9d7f9ba9f13f48d1b4709202bebe8d9f/1.jpg?token-time=1564704000&token-hash=DQMqNPtnn7BenunUTdHPPf6UlE30ajiHXiIn16FAM9g%3D HTTP/1.1" 200 1063004
./gallery-dl/patreon/2019_07_14_08_21_33_Cassandra_disguise_by_Vytz.jpg
[urllib3.connectionpool][debug] https://c10.patreonusercontent.com:443 "GET /3/e30%3D/patreon-media/p/post/28365274/1aae83f1a016451b913206d53ecf6f81/1.png?token-time=1564704000&token-hash=J4lq4S9Urlml5L9H3wWCfoIfbj1uTpj1Znahpbb9i6A%3D HTTP/1.1" 200 4866848
./gallery-dl/patreon/2019_07_14_08_18_47_Sonic_chain_unnoticed_prisoners.png
[urllib3.connectionpool][debug] https://c10.patreonusercontent.com:443 "GET /3/e30%3D/patreon-media/p/post/28218292/740e07f8bf474350a1e0926854e67753/1.jpg?token-time=1564704000&token-hash=0rkDVa2xEl2_U_pdcK_W90TGDIBUDY5_sPm20t977Ro%3D HTTP/1.1" 200 1884380
./gallery-dl/patreon/2019_07_08_11_33_28_Not_so_quiet_ambush_by_Vytz.jpg

Hm... everything works just fine on my machine.

Are you log in with username and password or session-id?

currently, you can only download your exclusives with your session-id.

mikf commented 5 years ago

The 403: Forbidden error comes from a Cloudflare CAPTCHA page. More infos here: https://github.com/mikf/gallery-dl/issues/227#issuecomment-489370855.

This was technically already fixed in 56c7a66, and 84f4d3bc0b85d7f434da272b7b000036430d9257 makes it work even without pyOpenSSL installed. Get version 1.9.0, which was released an hour ago, and accessing Patreon should work (again).

Aster-the-Med-Stu commented 5 years ago

Thanks, it works!

harvimt commented 4 years ago

This problem seems to be back...

> gallery-dl --version
1.12.3

Windows 10 64 bit, with pyopenssl and/or cryptography, upgraded all the dependencies to latest as well (requests etc.)

Logging into Patreon using a cookies.txt file

mikf commented 4 years ago

What's your Python version? You need Python 3.7.4 or higher on Windows.

See https://github.com/mikf/gallery-dl/issues/508#issuecomment-563414940

harvimt commented 4 years ago

yep that's it I was on 3.7.2 and upgraded to 3.8.0 and that fixed it.

On Wed, Jan 22, 2020 at 7:24 AM Mike Fährmann notifications@github.com wrote:

What's your Python version? You need Python 3.7.4 or higher on Windows.

See #508 (comment) https://github.com/mikf/gallery-dl/issues/508#issuecomment-563414940

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/mikf/gallery-dl/issues/342?email_source=notifications&email_token=AAA5CQVFIHT6LPLBUXWM3VLQ7BQKTA5CNFSM4IEWRFZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJT7MHQ#issuecomment-577238558, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA5CQUKET632GXAB7VJSLLQ7BQKTANCNFSM4IEWRFZA .