patreon: Cloudflare challenge and 403 Forbidden

[BosonHair]

Hello, Seeing some similar ones that are closed but still having issues.

Using: gallery_dl-1.26.2

[3/18] https://www.patreon.com/c/arion gallery-dl: Starting DownloadJob for 'https://www.patreon.com/c/arion' patreon: Using PatreonCreatorExtractor for 'https://www.patreon.com/c/arion' patreon: Loading cookies from '~/gallery-dl/cookies_patreon.txt' patreon: Sleeping 1.56 seconds (extractor) urllib3.connectionpool: Resetting dropped connection: www.patreon.com urllib3.connectionpool: https://www.patreon.com:443 "GET /arion HTTP/1.1" 403 None patreon: Cloudflare challenge patreon: HttpError: '403 Forbidden' for 'https://www.patreon.com/arion' patreon: Traceback (most recent call last): File "/home/USER/.pyenv/versions/3.10.1/lib/python3.10/site-packages/gallery_dl/job.py", line 151, in run for msg in extractor: File "/home/USER/.pyenv/versions/3.10.1/lib/python3.10/site-packages/gallery_dl/extractor/patreon.py", line 39, in items for post in self.posts(): File "/home/USER/.pyenv/versions/3.10.1/lib/python3.10/site-packages/gallery_dl/extractor/patreon.py", line 321, in posts campaign_id = self._get_campaign_id(creator, query) File "/home/USER/.pyenv/versions/3.10.1/lib/python3.10/site-packages/gallery_dl/extractor/patreon.py", line 347, in _get_campaign_id page = self.request(url, notfound="creator").text File "/home/USER/.pyenv/versions/3.10.1/lib/python3.10/site-packages/gallery_dl/extractor/common.py", line 243, in request raise exception.HttpError(msg, response) gallery_dl.exception.HttpError: '403 Forbidden' for 'https://www.patreon.com/arion'

My config: "patreon": { "cookies": "~/gallery-dl/cookiespatreon.txt", "tls12": false, "browser": false, "user-agent": "Patreon/72.2.28 (Android; Android 14; Scale/2.10)", "filename": "{date:%Y-%m-%d}{id}{title}{num:>02}.{extension}" },

Any idea?

[mikf]

gallery_dl-1.26.2

Try updating to the latest version. 1.26.2 is nearly one year old. (807ddde7e102f424ee3845f14450bf5f0c35fc79)

[BosonHair]

I'm sorry, I wrote it wrong. I'm using gallery-dl --version 1.28.0-dev

[mikf]

Could you try using a campaign ID URL like described in https://github.com/mikf/gallery-dl/issues/6348#issuecomment-2425681496 to avoid the request to https://www.patreon.com/arion ?

[BosonHair]

Sure thing:

gallery-dl "www.patreon.com/id:450124"
patreon: HttpError: '403 Forbidden' for 'https://www.patreon.com/api/posts?include=campaign,access_rules,attachments,attachments_media,audio,images,media,native_video_insights,poll.choices,poll.current_user_responses.user,poll.current_user_responses.choice,poll.current_user_responses.poll,user,user_defined_tags,ti_checks&fields%5Bcampaign%5D=currency,show_audio_post_download_links,avatar_photo_url,avatar_photo_image_urls,earnings_visibility,is_nsfw,is_monthly,name,url&fields%5Bpost%5D=change_visibility_at,comment_count,commenter_count,content,current_user_can_comment,current_user_can_delete,current_user_can_view,current_user_has_liked,embed,image,insights_last_updated_at,is_paid,like_count,meta_image_url,min_cents_pledged_to_view,post_file,post_metadata,published_at,patreon_url,post_type,pledge_url,preview_asset_type,thumbnail,thumbnail_url,teaser_text,title,upgrade_url,url,was_posted_by_campaign_owner,has_ti_violation,moderation_status,post_level_suspension_removal_date,pls_one_liners_by_category,video_preview,view_count&fields%5Bpost_tag%5D=tag_type,value&fields%5Buser%5D=image_url,full_name,url&fields%5Baccess_rule%5D=access_rule_type,amount_cents&fields%5Bmedia%5D=id,image_urls,download_url,metadata,file_name&fields%5Bnative_video_insights%5D=average_view_duration,average_view_pct,has_preview,id,last_updated_at,num_views,preview_views,video_duration&filter%5Bcampaign_id%5D=450124&filter%5Bcontains_exclusive_posts%5D=true&filter%5Bis_draft%5D=false&sort=-published_at&json-api-version=1.0'

Also did a print traffic for you:

gallery-dl "www.patreon.com/arion" --print-traffic
send: b'GET /arion HTTP/1.1\r\nHost: www.patreon.com\r\nUser-Agent: Patreon/72.2.28 (Android; Android 14; Scale/2.10)\r\nAccept: */*\r\nAccept-Language: en-US,en;q=0.5\r\nAccept-Encoding: gzip, deflate, br\r\nReferer: https://www.patreon.com/\r\nCookie: cf_clearance=COOKIE\r\n\r\n'
reply: 'HTTP/1.1 403 Forbidden\r\n'
header: Date: Mon, 28 Oct 2024 15:23:09 GMT
header: Content-Type: text/html; charset=UTF-8
header: Transfer-Encoding: chunked
header: Connection: close
header: Accept-CH: Sec-CH-UA-Bitness, Sec-CH-UA-Arch, Sec-CH-UA-Full-Version, Sec-CH-UA-Mobile, Sec-CH-UA-Model, Sec-CH-UA-Platform-Version, Sec-CH-UA-Full-Version-List, Sec-CH-UA-Platform, Sec-CH-UA, UA-Bitness, UA-Arch, UA-Full-Version, UA-Mobile, UA-Model, UA-Platform-Version, UA-Platform, UA
header: Critical-CH: Sec-CH-UA-Bitness, Sec-CH-UA-Arch, Sec-CH-UA-Full-Version, Sec-CH-UA-Mobile, Sec-CH-UA-Model, Sec-CH-UA-Platform-Version, Sec-CH-UA-Full-Version-List, Sec-CH-UA-Platform, Sec-CH-UA, UA-Bitness, UA-Arch, UA-Full-Version, UA-Mobile, UA-Model, UA-Platform-Version, UA-Platform, UA
header: Cross-Origin-Embedder-Policy: require-corp
header: Cross-Origin-Opener-Policy: same-origin
header: Cross-Origin-Resource-Policy: same-origin
header: Origin-Agent-Cluster: ?1
header: Permissions-Policy: accelerometer=(),autoplay=(),browsing-topics=(),camera=(),clipboard-read=(),clipboard-write=(),geolocation=(),gyroscope=(),hid=(),interest-cohort=(),magnetometer=(),microphone=(),payment=(),publickey-credentials-get=(),screen-wake-lock=(),serial=(),sync-xhr=(),usb=()
header: Referrer-Policy: same-origin
header: X-Content-Options: nosniff
header: X-Frame-Options: SAMEORIGIN
header: cf-mitigated: challenge
header: cf-chl-out: LONGCODE==
header: Cache-Control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
header: Expires: Thu, 01 Jan 1970 00:00:01 GMT
header: Set-Cookie: _COOKIE; path=/; expires=Mon, 28-Oct-24 15:53:09 GMT; domain=.patreon.com; HttpOnly; Secure; SameSite=None
header: Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=LONGCODE}],"group":"cf-nel","max_age":604800}
header: NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
header: Vary: Accept-Encoding
header: Strict-Transport-Security: max-age=2592000
header: X-Content-Type-Options: nosniff
header: Server: cloudflare
header: CF-RAY: 8d9becea9efda204-YYZ
header: Content-Encoding: gzip
patreon: Cloudflare challenge
patreon: HttpError: '403 Forbidden' for 'https://www.patreon.com/arion'