[BUG]: RSS Guard's user agent causes a 403 error with PsyPost's RSS Feed

mmkthecoolest commented 3 months ago

Brief description of the issue

RSS Guard cannot access https://www.psypost.org/ or its RSS feed due to its user agent having an outdated Chrome version number regardless of IP address. To be more specific, the Chrome version that RSS Guard uses for its user agent string is 87, which leads to an access denied error. However, changing the Chrome version to values within 111-199 inclusive fixes the issue. This can be verified using curl:

Using the original user agent string

curl -A "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) QtWebEngine/5.15.16 Chrome/87.0.4280.144 Safari/537.36 RSS Guard/4.6.6" https://www.psypost.org/feed/
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx</center>
</body>
</html>
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->

Changing the Chrome version to 111

curl -A "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) QtWebEngine/5.15.16 Chrome/111 Safari/537.36 RSS Guard/4.6.6" https://www.psypost.org/feed/
<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
    xmlns:content="http://purl.org/rss/1.0/modules/content/"
    xmlns:wfw="http://wellformedweb.org/CommentAPI/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:atom="http://www.w3.org/2005/Atom"
    xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
    xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
    >

<channel>
...
    </channel>
</rss>

How to reproduce the bug?

Add the RSS feed for PsyPost using the following link: https://www.psypost.org/feed/
Try to fetch articles from that source

What was the expected result?

The articles should be fetched with no errors

What actually happened?

The articles are not fetched and RSS Guard gives the status: network error (access to content was denied)

Debug log

rssguardlog.txt

Operating system and version

OS: Fedora Linux 39 (Using Flatpak)
RSS Guard version: 4.6.6

martinrotter commented 3 months ago

responded to you on discord:

RSS Guard allows to set CUSTOM user-agent via command line parameter --user-agent see rssguad.exe --help for more information i will update its build-int user-agent to newer version

skunkos — Today at 08:17 which variant of RSS Guard you use? Lite or regular with WebEngine? if regular you have to use the above switch

if lite I will update bundled user-agent or also use the switch

regular variant of RSS Guard uses dynamically constructed user-agent assembled Chromium and cannot be changed

guihkx commented 3 months ago

which variant of RSS Guard you use? Lite or regular with WebEngine?

Judging by the paths in his log file, he's using the Regular, Flatpak version.

regular variant of RSS Guard uses dynamically constructed user-agent assembled Chromium and cannot be changed

Wouldn't it be possible with this API?

martinrotter commented 3 months ago

martinrotter / rssguard