search

fossar / selfoss

multipurpose rss reader, live stream, mashup, aggregation web application
https://selfoss.aditu.de
GNU General Public License v3.0
2.35k stars 343 forks source link

Update not thread safe anymore #1398

Closed michix closed 1 year ago

michix commented 1 year ago

It seems to me that with version 2.19 the /update is not thread safe anymore: when called multiple times in sequence, entries show up twice and the load on the server increases significantly. Before this version this never happened.

jtojnar commented 1 year ago

This has always been the case. See #208, https://github.com/fossar/selfoss/issues/921 and https://github.com/fossar/selfoss/issues/967. There is a partial fix in https://github.com/fossar/selfoss/pull/995 but it is not ready.

For now, I would recommend not running /update multiple times. Or at least safeguarding against it with e.g. flock: https://github.com/fossar/selfoss/issues/967#issuecomment-319159139.

michix commented 1 year ago

Many thanks. The weird thing is that this never happened before in my installation, only when I upgraded to 2.19. But the root cause in my case was that I called the /update URL with wget and without --tries=1 which led to a lot of processes. This problem then got accelerated with loading the RSS of Google Project Zero with "RSS Fulltext" which went into an infinite loop due to parsing issues, I guess.

Whatever, I was not aware that /update is not thread safe and did not encounter this problem before, but since this is the way it is, I will close this issue.

Many thanks again!

jtojnar commented 1 year ago

There definitely should not be any infinite loops when loading RSS feeds. How did you determined there to be one?

Are you referring to https://googleprojectzero.blogspot.com/feeds/posts/default with feed type RSS (with content extraction)?

michix commented 1 year ago

Yes, I think that was it. Then I took "with enclosure" and it is fine now.