search

gugray / rss-parrot

Notifies Mastodon accounts about new posts in the RSS feeds they follow
https://rss-parrot.net
MIT License
109 stars 7 forks source link

Feed auto-discovery on websites does not always work #27

Closed kvibber closed 7 months ago

kvibber commented 7 months ago

Every time I've entered a website address that I know has <link>-based autodiscovery, I've gotten the "Hm, I can't find a feed for this site." response from the bot. Some examples:

https://wordpress.org/news/ (and every other WordPress site I've tried) https://kvibber.com/

It has no problem if I ask it for the feed URLs, but it's not finding the <link rel="alternate"... feeds for some reason.

gugray commented 7 months ago

With kvibber.com, the request timed out in 10 seconds. I've seen this with a few sites; it looks like there is something between the parrot's datacenter and the website that blocking the request. I'm not sure how to diagnose this further or what I may be able to change in the parrot so it's blocked less often...

Any info you can share about kvibber.com's hosting that might help here?

kvibber commented 7 months ago

Odd... it's a 12KB static html file (4KB compressed) served by Apache on a Dreamhost VPS with a Let's Encrypt TLS certificate. I can't think why it would time out unless the server was overloaded by something else at the time.

And the replies from birb have been fast - within a second. I tested it again while tailing the website server logs, and I got the following line showing up immediately after hitting Publish on the Mastodon DM:

64.176.202.191 - - [11/Jan/2024:08:49:30 -0800] "GET / HTTP/2.0" 200 4364 "-" "RSS-Parrot-Bot/0.0.44 (+https://rss-parrot.net)"

followed just as quickly with the reply from birb saying it couldn't find a feed.

So I don't think timeouts are the only issue.

Especially since I see the same issue when asking birb for other website URLs that have <link> elements for their feeds, like

https://isc.sans.edu/ https://news.indieweb.org/en https://www.11ty.dev/blog/

and so on.

gugray commented 7 months ago

OK, so this is apparently one of those facepalm moments 🤦

When I added that timeout only over a day ago, I made an annoying mistake and it wasn't in seconds but... millseconds. I just rolled out a fix; can you check again?

kvibber commented 7 months ago

Aha, that would do it! :laughing:

Confirmed: I requested the Eleventy blog and my own site and RSS Parrot was able to find both feeds immediately.

Thanks for the quick response!