Cloudflare Prevents fetching RSS feeds?

[ArtexJay]

Describe the bug RSBN and Donbass Insider both use Cloudflare to check if the connection is secure, I think to protect from DDOS attacks. I believe that due to this Handy is unable to fetch news from their respective feeds.

To Reproduce Steps to reproduce the behavior:

1. Add "https://www.rsbnetwork.com/category/news/feed/" or "https://www.donbass-insider.com/category/news/feed/"
2. Try to load the feed
3. See error

Expected behavior I expect Handy to load the articles as normal. Perhaps there needs to be a way to allow for the loading/redirect when this happens. However both feeds provide different errors, RSBN gives a Parse Error "At line 1 column 0: no element found" and Donbass Insider error is "Feed website is unreachable" even though I am able to load it in the web browser after having the cloudflare check

Smartphone (please complete the following information):

• Device: Xiaomi mi 9t Pro
• OS: Android 11
• Version 0.19.16

[ArtexJay]

Update: RSBN RSS feed just started to work again. However the other one I mentioned is still broken.

Close this thread if there is nothing that Handy can do.

[ArtexJay]

Update: The New American website has recently been updated with cloudflare protection as well, as a result Handy is unable to fetch articles for this feed as well feed https://thenewamerican.com/feed/ Web browser is able to load the website fine after the cloudflare DDOS check.

[touwys]

Update: The New American website has recently been updated with cloudflare protection as well, as a result Handy is unable to fetch articles for this feed as well feed https://thenewamerican.com/feed/ Web browser is able to load the website fine after the cloudflare DDOS check.

I concur, and I wonder a solution can be had? This is another Cloudflare-protected site: https://tnc.news/

[ArtexJay]

I concur, and I wonder a solution can be had? This is another Cloudflare-protected site: https://tnc.news/

Im not sure about you, but for me Handy is able to retrieve articles from True North with the feed https://tnc.news/feed/ Do you get the error "Feed website is unreachable"

[touwys]

Im not sure about you, but for me Handy is able to retrieve articles from True North with the feed https://tnc.news/feed/

Thanks, mine does not.

Do you get the error "Feed website is unreachable"

Yes:

[yanus171]

Cant find solution. I have checked https://stackoverflow.com/questions/32232259/accessing-webpage-with-cloudflare-protection?noredirect=1 and https://stackoverflow.com/questions/40055722/bypass-cloudflare-protected-page-and-get-html-source-in-java?noredirect=1

[ArtexJay]

Cant find solution. I have checked https://stackoverflow.com/questions/32232259/accessing-webpage-with-cloudflare-protection?noredirect=1 and https://stackoverflow.com/questions/40055722/bypass-cloudflare-protected-page-and-get-html-source-in-java?noredirect=1

Thats unfortunate then, would it be at all possible for Handy to bypass cloudflare by opening a webpage dialogue for the user and allowing cloudflare to do its thing and the get the feed once cloudflare has forwarded the user to the existing RSS link? Im not sure how feasible it is or if its even possible.

[ArtexJay]

It looks like other RSS readers are facing similar issues with accessing Cloudflare protected websites such as FreshRSS, QuiteRSS. But this is obviously a Cloudflare issue and not the RSS reader problem

[touwys]

ChatGPT, in response to my prompt, suggested this course of action:

As of my last update, Cloudflare does implement security measures that could potentially interfere with the functioning of RSS News readers when they try to access Cloudflare-protected web pages. The reason is Cloudflare's protective measures against bots, which can sometimes mistakenly identify RSS feed fetchers as malicious bots.

Cloudflare provides various security levels that a website owner can configure, and this configuration can impact how RSS feed readers are treated. If website owners find that legitimate services like RSS readers are being blocked, they can adjust their Cloudflare settings accordingly.

Here are a few potential solutions that website owners can implement:

1. Whitelisting User Agents: Website owners can whitelist the specific user agents used by popular RSS readers so that Cloudflare knows to allow them through.

2. Page Rules: Cloudflare allows the creation of Page Rules that can bypass certain security features for specific URLs or patterns. A website owner might set up a Page Rule to disable security features for their RSS feed URL.

3. Lowering Security Level: The overall security level set in Cloudflare can be lowered, but this could potentially expose the site to more security threats, so it's not generally recommended unless for specific URLs like the RSS feed.

4. IP Access Rules: If the RSS reader service uses a consistent range of IP addresses, those IPs can be whitelisted so that they are not challenged by Cloudflare's security mechanisms.

For RSS news reader services, the onus is typically on the service or the user to contact the owner of the website if they find themselves unable to access a feed due to Cloudflare's security features. Then, it's up to the website owner to adjust their Cloudflare settings if they choose to.

If you are the owner of a website and have this issue, or you are an RSS reader user and your favorite feeds are being blocked, the best approach is to reach out to the website's support or their technical team to inform them about the issue so that they can make the necessary adjustments in their Cloudflare settings.

---

For interest sake:

I have contacted several Cloudflare-protected website owners about the CloudFlare issue, and not one of them ever responded — not even after repeated enquiries.