[rss] Make feed refresh interval configurable for users

[kr428]

Currently, it seems Friendica does refresh RSS feeds then and now, but at "random" intervals for a user perspective. While this is sufficient for using Friendica as a feed reader, it feels a bit clumsy if using RSS for resharing posts off ones own blog. Would it be possible, for these use cases, to introduce a feed-specific time setting so users can make sure such feeds are updated more often?

[annando]

It is an admin setting to enable the possibility for a user to set their own poll intervals. But the automatic mode should work reliably. It polls in a frequency that matches the post frequency.

[kr428]

It polls in a frequency that matches the post frequency.

I am unsure I understand this, to be honest. Tried this with my personal blog to my Friendica instance, and I noticed it took in between 1..2h for a post to make it to my profile. Compared that, too, to Hubzilla on some system where this delay is somewhere in between 10..15mins (or micro.blog where it's even shorter, something in between 2..5mins). What exactly does this depend on?

[annando]

The system has got a look at the post frequency in the feed. So when this feed normally publishes a post once per day, then it will only query for the post once per day but not every 5 minutes.

[foss-]

The described behavior sounds sane to find a balance between load and good update interval. Does this need changing?

[kr428]

Without knowing how the system works under the hood, I still think it would be preferrable to set this interval manually per feed. Like, in example, my "micro blog" (external Bludit) updates fairly often while my long-form blog rarely sees new posts. The latter, I'd set to refresh, like, once a day or hour, while the former I'd probably set to a 10..15min interval. At the moment, this still seems considerably longer.

[annando]

Problem is that this creates a lot of unnecessary load in the system when a feed is updated maybe once a month but the user decided to update it every 5 minutes. Especially on systems that fetch a lot of feeds, this really can stress these systems. That's the reason why an admin can decide to set this to automatic for their users.

[kr428]

Yeah, I can imagine. That's why I have a difficult gut feeling about this one as well. What about adding some sort of webhook then? If an external system updates (its data and subsequently feed), maybe it would work to "notify" a Friendica instance to refresh "now", which would make it possible to have long polling intervals? Or is this too specific a use case?

[annando]

Once there had been a standard named PubSubHubbub. I'm unsure if anyone supports it in the wild.

[kr428]

Does Friendica support that at the moment? If so, I'd look into that and see whether that fits my needs. For looking at tools such as EchoFeed, they usually support a "plain" HTTP endpoint to post "something" to trigger "some" action (which seems both pragmatic and disputable approach).

[annando]

AFAIK it is only coded for OStatus. Please check if you find any feeds that support this standard. Then I can do some tests.