Almost all feeds stopped updating after NC & news upgrade

[xdmx]

• [X] I have read the CONTRIBUTING.md and followed the provided tips
• [X] I accept that the issue will be closed without comment if I do not check here
• [X] I accept that the issue will be closed without comment if I do not fill out all items in the issue template.

Explain the Problem

I have a Nextcloud instance on a third party provider, and the other day they upgraded it from v19 (I believe v19.0.4 or v19.0.6) to v20.0.4, that also upgraded News to the latest version, 15.3.0, which I then upgraded to 15.3.1. With both those versions almost all my feeds are not updating anymore, I just get a couple of them, but all the rest (more than 200) have completely stop, even though I'm sure there are new articles.

My News configuration hasn't changed (it's the default one), and the system cron shows that it's correctly working every 5 minutes, as otherwise I wouldn't get any new article at all.

Because I don't have access to the server I cannot inspect it deeply, but the Nextcloud logging page is not showing any error.

Is this a known issue or is there a way to fix it? I also think I cannot downgrade to a previous version from the Nextcloud admin panel.

Steps to Reproduce

Explain what you did to encounter the issue

1. I didn't nothing, with v19 it was working, then with v20 it stopped

System Information

• News app version: 15.3.1
• Nextcloud version: 20.0.4
• Cron type: (system cron/python updater/...) system
• PHP version: 7.4.13
• Database and version: mysql 10.3.27
• Browser and version: Firefox 85
• OS and version: (server) Linux 5.4.0-56-generic x86_64 / (desktop) Arch

Contents of nextcloud/data/nextcloud.log

N/A

Contents of Browser Error Console

``` Content Security Policy: The page’s settings blocked the loading of a resource at inline (“script-src”). injectGlobalHook.js:808:49 Content Security Policy: The page’s settings blocked the loading of a resource at inline (“script-src”). 2 content-script.js:40:65 Content Security Policy: The page’s settings blocked the loading of a resource at eval (“script-src”). angular.min.js:205 ```

[SMillerDev]

We had some reports around 15.0 that articles wouldn't update for a day and that might be the case here too. At the time we suspected it was due to the change in cron system, but since it went away after 24 hours we sort of just left it.

If you still have nothing after 24 hours, see if toggling full-text fetches the new articles.

[xdmx]

Thank you Sean, I've tried to enable it on 2 feeds, I'll keep you updated if that fixes it.

Something unrelated that I've noticed is that after I enabled the full text, both feeds lost their category and I had to put them back where they were, that'd be painful to enable all feeds and set them in categories as well :grimacing:

[xdmx]

It's been more than 48h-72h now since the upgrade and it hasn't started syncing them :disappointed:

Nothing changed also for those 2 feeds on which I enabled the full index.

I've also tried to add a new feed and it initially fetched the articles, but then it didn't updated anymore either.

What I don't understand is why a very few number of existing feeds are getting updated, while the rest is not :man_shrugging:

[Grotax]

So a few feeds are getting updated automatically and the rest don't?

It would probably be helpful if you provide some of the failing feeds.

Also you can check them with https://feed-io.net/ if you want.

There is/was this know issue that updates would get blocked completely probably a race condition.

But single feeds updating, while others don't sounds strange especially if you don't find any errors in your log maybe also something else changes in your environment.

Also take note that news isn't directly bound to the cron system so it's not updated every 5 minutes, instead it hooks into the job system and can the interval can be configured in the admin settings.

[xdmx]

So a few feeds are getting updated automatically and the rest don't?

Correct, so far the majority are not, just very few are getting updated (at least 2 for sure)

One that is failing is http://hnapp.com/rss?q=score%3E32, which I'm 100% sure there are new posts pretty often. And https://www.hetzner-status.de/en.atom as well. For both it shows the last article from 3 days ago, not the latest ones.

While https://weworkremotely.com/remote-jobs.rss and https://discourse.stimulusjs.org/posts.rss are the ones that are correctly updating.

I've tried all 4 on feed-io.net, but it's not fetching any of them and I'm getting a CORS Missing Allow Origin error in Firefox and a net::ERR_FAILED in Opera and Chromium. Even for those that are correctly updating. I've also tried after disabling all browser extensions.

Firefox requests & HAR

 ``` { "log": { "version": "1.2", "creator": { "name": "Firefox", "version": "85.0" }, "browser": { "name": "Firefox", "version": "85.0" }, "pages": [ { "startedDateTime": "2021-02-07T16:55:26.998+01:00", "id": "page_1", "title": "Feed-io by alexdebril", "pageTimings": { "onContentLoad": 754, "onLoad": 781 } } ], "entries": [ { "pageref": "page_1", "startedDateTime": "2021-02-07T16:55:26.998+01:00", "request": { "bodySize": 42, "method": "POST", "url": "https://api.feed-io.net/feed/consume", "httpVersion": "HTTP/1.1", "headers": [ { "name": "Host", "value": "api.feed-io.net" }, { "name": "User-Agent", "value": "Mozilla/5.0 (X11; Linux x86_64; rv:85.0) Gecko/20100101 Firefox/85.0" }, { "name": "Accept", "value": "*/*" }, { "name": "Accept-Language", "value": "en-US,en;q=0.5" }, { "name": "Accept-Encoding", "value": "gzip, deflate, br" }, { "name": "Referer", "value": "https://feed-io.net/" }, { "name": "Content-Type", "value": "text/plain;charset=UTF-8" }, { "name": "Content-Length", "value": "42" }, { "name": "Origin", "value": "https://feed-io.net" }, { "name": "DNT", "value": "1" }, { "name": "Connection", "value": "keep-alive" }, { "name": "Pragma", "value": "no-cache" }, { "name": "Cache-Control", "value": "no-cache" } ], "cookies": [], "queryString": [], "headersSize": 413, "postData": { "mimeType": "text/plain;charset=UTF-8", "params": [], "text": "{\"url\": \"http://hnapp.com/rss?q=score>32\"}" } }, "response": { "status": 200, "statusText": "OK", "httpVersion": "HTTP/1.1", "headers": [ { "name": "Server", "value": "nginx/1.19.6" }, { "name": "Content-Type", "value": "application/json" }, { "name": "Transfer-Encoding", "value": "chunked" }, { "name": "Connection", "value": "keep-alive" }, { "name": "X-Powered-By", "value": "PHP/8.0.1" }, { "name": "Cache-Control", "value": "no-cache, private" }, { "name": "Date", "value": "Sun, 07 Feb 2021 15:55:27 GMT" }, { "name": "Content-Encoding", "value": "gzip" } ], "cookies": [], "content": { "mimeType": "application/json", "size": 0, "text": "" }, "redirectURL": "", "headersSize": 245, "bodySize": 700 }, "cache": {}, "timings": { "blocked": -1, "dns": 0, "connect": 0, "ssl": 0, "send": 0, "wait": 243, "receive": 0 }, "time": 243, "_securityState": "secure", "serverIPAddress": "51.255.170.34", "connection": "443" }, { "pageref": "page_1", "startedDateTime": "2021-02-07T16:57:45.147+01:00", "request": { "bodySize": 53, "method": "POST", "url": "https://api.feed-io.net/feed/consume", "httpVersion": "HTTP/1.1", "headers": [ { "name": "Host", "value": "api.feed-io.net" }, { "name": "User-Agent", "value": "Mozilla/5.0 (X11; Linux x86_64; rv:85.0) Gecko/20100101 Firefox/85.0" }, { "name": "Accept", "value": "*/*" }, { "name": "Accept-Language", "value": "en-US,en;q=0.5" }, { "name": "Accept-Encoding", "value": "gzip, deflate, br" }, { "name": "Referer", "value": "https://feed-io.net/" }, { "name": "Content-Type", "value": "text/plain;charset=UTF-8" }, { "name": "Content-Length", "value": "53" }, { "name": "Origin", "value": "https://feed-io.net" }, { "name": "DNT", "value": "1" }, { "name": "Connection", "value": "keep-alive" }, { "name": "Pragma", "value": "no-cache" }, { "name": "Cache-Control", "value": "no-cache" } ], "cookies": [], "queryString": [], "headersSize": 413, "postData": { "mimeType": "text/plain;charset=UTF-8", "params": [], "text": "{\"url\": \"https://weworkremotely.com/remote-jobs.rss\"}" } }, "response": { "status": 200, "statusText": "OK", "httpVersion": "HTTP/1.1", "headers": [ { "name": "Server", "value": "nginx/1.19.6" }, { "name": "Content-Type", "value": "application/json" }, { "name": "Transfer-Encoding", "value": "chunked" }, { "name": "Connection", "value": "keep-alive" }, { "name": "X-Powered-By", "value": "PHP/8.0.1" }, { "name": "Cache-Control", "value": "no-cache, private" }, { "name": "Date", "value": "Sun, 07 Feb 2021 15:57:46 GMT" }, { "name": "Content-Encoding", "value": "gzip" } ], "cookies": [], "content": { "mimeType": "application/json", "size": 0, "text": "" }, "redirectURL": "", "headersSize": 245, "bodySize": 711 }, "cache": {}, "timings": { "blocked": 5, "dns": 50, "connect": 35, "ssl": 57, "send": 0, "wait": 1026, "receive": 0 }, "time": 1173, "_securityState": "secure", "serverIPAddress": "51.255.170.34", "connection": "443" }, { "pageref": "page_1", "startedDateTime": "2021-02-07T16:58:08.338+01:00", "request": { "bodySize": 37, "method": "POST", "url": "https://api.feed-io.net/feed/discover", "httpVersion": "HTTP/1.1", "headers": [ { "name": "Host", "value": "api.feed-io.net" }, { "name": "User-Agent", "value": "Mozilla/5.0 (X11; Linux x86_64; rv:85.0) Gecko/20100101 Firefox/85.0" }, { "name": "Accept", "value": "*/*" }, { "name": "Accept-Language", "value": "en-US,en;q=0.5" }, { "name": "Accept-Encoding", "value": "gzip, deflate, br" }, { "name": "Referer", "value": "https://feed-io.net/" }, { "name": "Content-Type", "value": "text/plain;charset=UTF-8" }, { "name": "Content-Length", "value": "37" }, { "name": "Origin", "value": "https://feed-io.net" }, { "name": "DNT", "value": "1" }, { "name": "Connection", "value": "keep-alive" }, { "name": "Pragma", "value": "no-cache" }, { "name": "Cache-Control", "value": "no-cache" } ], "cookies": [], "queryString": [], "headersSize": 414, "postData": { "mimeType": "text/plain;charset=UTF-8", "params": [], "text": "{\"url\": \"https://weworkremotely.com\"}" } }, "response": { "status": 200, "statusText": "OK", "httpVersion": "HTTP/1.1", "headers": [ { "name": "Server", "value": "nginx/1.19.6" }, { "name": "Content-Type", "value": "application/json" }, { "name": "Transfer-Encoding", "value": "chunked" }, { "name": "Connection", "value": "keep-alive" }, { "name": "X-Powered-By", "value": "PHP/8.0.1" }, { "name": "Cache-Control", "value": "no-cache, private" }, { "name": "Date", "value": "Sun, 07 Feb 2021 15:58:09 GMT" }, { "name": "Content-Encoding", "value": "gzip" } ], "cookies": [], "content": { "mimeType": "application/json", "size": 0, "text": "" }, "redirectURL": "", "headersSize": 245, "bodySize": 696 }, "cache": {}, "timings": { "blocked": -1, "dns": 0, "connect": 0, "ssl": 0, "send": 0, "wait": 1285, "receive": 0 }, "time": 1285, "_securityState": "secure", "serverIPAddress": "51.255.170.34", "connection": "443" }, { "pageref": "page_1", "startedDateTime": "2021-02-07T16:58:25.500+01:00", "request": { "bodySize": 53, "method": "POST", "url": "https://api.feed-io.net/feed/consume", "httpVersion": "HTTP/1.1", "headers": [ { "name": "Host", "value": "api.feed-io.net" }, { "name": "User-Agent", "value": "Mozilla/5.0 (X11; Linux x86_64; rv:85.0) Gecko/20100101 Firefox/85.0" }, { "name": "Accept", "value": "*/*" }, { "name": "Accept-Language", "value": "en-US,en;q=0.5" }, { "name": "Accept-Encoding", "value": "gzip, deflate, br" }, { "name": "Referer", "value": "https://feed-io.net/" }, { "name": "Content-Type", "value": "text/plain;charset=UTF-8" }, { "name": "Content-Length", "value": "53" }, { "name": "Origin", "value": "https://feed-io.net" }, { "name": "DNT", "value": "1" }, { "name": "Connection", "value": "keep-alive" }, { "name": "Pragma", "value": "no-cache" }, { "name": "Cache-Control", "value": "no-cache" } ], "cookies": [], "queryString": [], "headersSize": 413, "postData": { "mimeType": "text/plain;charset=UTF-8", "params": [], "text": "{\"url\": \"https://weworkremotely.com/remote-jobs.rss\"}" } }, "response": { "status": 200, "statusText": "OK", "httpVersion": "HTTP/1.1", "headers": [ { "name": "Server", "value": "nginx/1.19.6" }, { "name": "Content-Type", "value": "application/json" }, { "name": "Transfer-Encoding", "value": "chunked" }, { "name": "Connection", "value": "keep-alive" }, { "name": "X-Powered-By", "value": "PHP/8.0.1" }, { "name": "Cache-Control", "value": "no-cache, private" }, { "name": "Date", "value": "Sun, 07 Feb 2021 15:58:26 GMT" }, { "name": "Content-Encoding", "value": "gzip" } ], "cookies": [], "content": { "mimeType": "application/json", "size": 0, "text": "" }, "redirectURL": "", "headersSize": 245, "bodySize": 711 }, "cache": {}, "timings": { "blocked": -1, "dns": 0, "connect": 0, "ssl": 0, "send": 0, "wait": 946, "receive": 0 }, "time": 946, "_securityState": "secure", "serverIPAddress": "51.255.170.34", "connection": "443" } ] } } ```

This is the Nextcloud logging page, totally empty:

Also take note that news isn't directly bound to the cron system so it's not updated every 5 minutes, instead it hooks into the job system and can the interval can be configured in the admin settings.

Well, the system cron is set to run every 5 minutes (and it is), while News is set to update every hour, which has already been passed by many hours :grimacing:

I'm not sure what else I could access / provide you to help debugging this, not having server or mysql access definitely doesn't help here :disappointed:

In the meantime I've tried to export all feeds and import them in Akregator and they're updating (just to double check that they're publishing new articles)

[Grotax]

Yea ok I checked your feeds they work with feed-io, I will add them to my problem folder and check if they get updated for me.

[xdmx]

Thank you

As an extra test I've added some of those feeds and different/new ones as well on a different user account (on the same nextcloud instance), but it's still not updating them :disappointed:

[Grotax]

That doesn't change anything as the update works the same for every user. Probably your database is screwed or your server has connection issues.

It's weird that you have no logs at all.

And that you can't access the dB is kind of not good for hosting nextcloud, it's not there yet at least not with 3d party apps. It's still needed that the admin checks the dB sometimes or runs a query. Some things can be done through OCC but if you can't even access that it doesn't help much.

[xdmx]

I wish Nextcloud had some kind of Admin page (or app) for exploring the databse through raw sql, even if it was just read only, that could help in these cases (unless there is some kind of app sandboxing for security reasons, I don't know).

I guess I'll wait few more days and then try to write to the provider to see if they can do something.

Any suggestion on what to check if there was access to the server / database?

[Grotax]

Yea there is in the faq a part about feeds not updating including SQL statements https://github.com/nextcloud/news/blob/master/docs/faq/README.md#feeds-not-updated

[xdmx]

Just got a reply from them and it seems that it was caused by https://github.com/nextcloud/news/issues/859 and they were able to fix it. Now feeds are updating again :tada: