search

daemionfox / patreon-feed

Patreon RSS library based on Patreon's API
41 stars 6 forks source link

Patreon added Cloudflare, blocks get_file_contents #7

Closed TwistedMexi closed 3 years ago

TwistedMexi commented 5 years ago

This needs to be replaced with curl and valid useragent headers I believe. Cloudflare is rejecting it as a bad request now.

daemionfox commented 5 years ago

Yeah, I haven't found the way around Cloudflare's rules yet. cURL would be the next step. I started looking at using https://github.com/KyranRana/cloudflare-bypass to try and get through, but haven't figured out the workaround.

If you want to take a shot at it and open up a pull-request, I'm all for it.

TwistedMexi commented 5 years ago

It's definitely the user-agent missing for the initial issue, but I haven't figured out how to get CURL to follow the 301 redirect after. (haven't spent much time on it yet). I'll see what I can do.

daemionfox commented 5 years ago

Might be that the 3** needs to be captured and another cURL then sent out as a followup? Don't mind me, just brainstorming without actually thinking.

Roliga commented 5 years ago

I've been implementing a Patreon feed in rss-bridge and I found that if you use cURL you can do some trickery to get around this protection. I had to set the Accept-Language header to en-US and make the cipher list I guess not look like cURL/look more like a web browser by removing the DHE-RSA-CHACHA20-POLY1305 cipher. For that you can do something like:

curl_setopt($curl_resource, CURLOPT_SSL_CIPHER_LIST, 'DEFAULT:!DHE-RSA-CHACHA20-POLY1305');

A non-standard user agent may also be necessary but I'm not certain of that.

See here for how I did it. That also has some links to other people who are dealing with this.

splitbrain commented 5 years ago

(Trying to implement this in my own project) I can't get this to work using stream contexts. I could rewrite this to use CURL but trying to avoid CloudFlare's bot detection is probably playing cat'n'mouse forever.

Users should petition to Patreon getting their shit together.

See also the discussion here: https://www.patreondevelopers.com/t/feature-suggestion-acquire-patreon-posts/462/12

Bluscream commented 4 years ago

Can we get this fixed?

daemionfox commented 3 years ago

I've released a new version (v.3) that covers everything and adds a bunch of new stuff. That's the good news. The bad news, it now requires a Patreon API Access Token, and you can only retrieve information from the account associated with that access token.