search

dakboard / Cloud-Platform

Feature requests, enhancements and anything you'd like to see in DAKboard!
https://dakboard.com
168 stars 39 forks source link

User-Agent causing issues with some RSS feeds #2068

Closed rsalsbery closed 1 month ago

rsalsbery commented 8 months ago

It has come to our attention that some RSS feeds that have been recently updated by their host are having issues with the User-Agent header we send for their requests. This is preventing us from being authorized to access their data. This case is being created to test updates to the User-Agent to make sure it will not cause issues with the existing feeds that are still working if we update it.

Feeds that are currently incompatible with useragent:

Update

After some discussion, the approach for this case will now become code that can auto-retry a small set (3-4) of dependable User Agent strings. When one succeeds, it can be saved to the block configuration for re-use going forward.

cocarrig commented 1 month ago

Pursuant to recent design updates for RSS, determine if this is still and issue.

Dan-Peck commented 1 month ago

The UA adjustment for the Dutch site has been handled via work done on #2192.

However, shifting this to a different approach, where we can auto-retry around 3-4 User Agent strings during the RSS link validation phase. If one succeeds, that can be saved for re-use with the block from that point on.

Dan-Peck commented 1 month ago

Testing a wide array of RSS links has yielded successful results with existing code. We can address any issues if they arise in the future; for now this issue is being considered resolved.