[x] I have read the CONTRIBUTING.md and followed the provided tips
[x] I accept that the issue will be closed without comment if I do not check here
[x] I accept that the issue will be closed without comment if I do not fill out all items in the issue template.
Explain the Problem
Since I updated NextCloud and also the News app most of my feeds aren't updated anymore. This seems to be caused by this feed: http://www.windytan.com/feeds/posts/default
It is a valid Atom 1.0 feed. feed-io failed for me, because Firefox denied the "consume" request with "CORS Missing Allow Origin". But i think fetching is fine, as an exception is logged in nextcloud.log only when inserting the data to the database. This probably leads to all following feeds not being updated.
This started with my update of NextCloud and News. This is a grep on "read access" in the log showing this clearly:
... even more feeds in this cron call
{"reqId":"iYCN1BkNdPgjQ86n5CSj","level":0,"time":"2021-01-23T21:34:48+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : https://foonathan.net/post/index.xml into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.4.0"}
{"reqId":"iYCN1BkNdPgjQ86n5CSj","level":0,"time":"2021-01-23T21:34:48+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : https://github.com/kimai/kimai/releases.atom into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.4.0"}
{"reqId":"iYCN1BkNdPgjQ86n5CSj","level":0,"time":"2021-01-23T21:34:48+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : https://blog.linuxmint.com/?feed=rss2 into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.4.0"}
{"reqId":"iYCN1BkNdPgjQ86n5CSj","level":0,"time":"2021-01-23T21:34:48+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : https://blog.gitea.io/index.xml into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.4.0"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:20+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : http://www.tagesschau.de/xml/rss2 into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:20+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : http://www.windytan.com/feeds/posts/default into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.5.2"}
{"reqId":"KGIXbcHcmeEDp796Z9Mf","level":0,"time":"2021-01-23T23:34:27+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : http://www.tagesschau.de/xml/rss2 into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.5.2"}
{"reqId":"KGIXbcHcmeEDp796Z9Mf","level":0,"time":"2021-01-23T23:34:27+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : http://www.windytan.com/feeds/posts/default into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.5.2"}
System Information
News app version: 15.3.1
Nextcloud version: 20.0.5
Cron type: systemd cron.php
PHP version: 7.4.14
Database and version: MariaDB 10.3.27
Browser and version: Firefox 78.7.0
OS and version: Debian 10
Contents of nextcloud/data/nextcloud.log
```json
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:20+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : http://www.windytan.com/feeds/posts/default into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:20+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"start reading http://www.windytan.com/feeds/posts/default","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":1,"time":"2021-01-23T22:34:20+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"no 'modifiedSince' parameter given, setting it to 01/01/1970","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":1,"time":"2021-01-23T22:34:20+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"hitting http://www.windytan.com/feeds/posts/default","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"response ok, now turning it into a document","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":1,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"the stream is modified, parsing it","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"accurate parser : FeedIo\\Parser\\XmlParser","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"Feed http://www.windytan.com/feeds/posts/default was modified since last fetch. #25 items","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"Added item Plotting patterns in music with a fantasy record player for feed absorptions publishdate: 1608189512","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"Added item Capturing PAL video with an SDR (and a few dead-ends) for feed absorptions publishdate: 1589710223","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"Added item Beeps and melodies in two-way radio for feed absorptions publishdate: 1566587018","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":3,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"core","method":"","url":"--","message":{"Exception":"Doctrine\\DBAL\\Exception\\DriverException","Message":"An exception occurred while executing 'UPDATE `oc_news_items` SET `unread` = ?, `url` = ?, `guid` = ?, `guid_hash` = ?, `pub_date` = ?, `last_modified` = ?, `title` = ?, `author` = ?, `body` = ?, `search_index` = ?, `fingerprint` = ?, `content_hash` = ?, `feed_id` = ? WHERE `id` = ?' with params [false, \"http:\\/\\/www.windytan.com\\/2014\\/10\\/visualizing-hex-bytes-with-unicode-emoji.html\", \"tag:blogger.com,1999:blog-5096278891763426276.post-7039745451118618656\", \"a47b16a321e3f78a36d0ab89e53e505b\", 1519239491, \"1611441261931408\", \"Visualizing hex dumps with Unicode emoji\", \"Oona R\\u00e4is\\u00e4nen\", \"
Memorizing SSH public key fingerprints can be difficult; they're just long random numbers displayed in base 16. There are some terminal-friendly solutions, like OpenSSH's randomart. But because I use a Unicode terminal, I like to map the individual bytes into characters in the Miscellaneous Symbols and Pictographs<\\/a> block.<\\/p>
And you are right, fetching the feed is no problem I tried it on cli with feed-io.
The issue happens when news tries to insert the data into the DB, I have to say though that I don't understand the error message.
SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\\xF0\\x9F\\x8C\\x80 ...' for column `newown_nordlichter_brv_de`.`oc_news_items`.`body` at row 1",
Memorizing SSH public key fingerprints can be difficult; they're just long random numbers displayed in base 16. There are some terminal-friendly solutions, like OpenSSH's randomart. But because I use a Unicode terminal, I like to map the individual bytes into characters in the Miscellaneous Symbols and Pictographs<\\/a> block.<\\/p>
Probably the source of the error lies in the body of the post "Animated line drawings with OpenCV", as the previous posts were added (according to the nextcloud.log).
For me it seems to be valid UTF-8, I could not find any chars apart [-a-zA-Z0-9<>,.;:-_"\/\\+#\'&(){}*%$!|] in it. Perhaps something is interpreted incorrectly due to some \s?
As a temporary workaround I set the prevent_update flag in the oc_news_feeds to 1 for this feed. Now all the other feeds are successfully fetched again.
{"reqId":"uatNiR4c0l1uWj2bT5fY","level":3,"time":"2021-02-23T15:30:03+00:00","remoteAddr":"","user":"--","app":"core","method":"","url":"--","message":{"Exception":"Doctrine\\DBAL\\Exception\\DriverException","Message":"An exception occurred while executing 'INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)' with params [true, \"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\", \"t3_lqglv5\", \"ae4638099ccceaabee9bb6828307d5a1\", 1614081022, \"1614094203478929\", \"\\u23f1\\ufe0f \\u23f3 \\ud83c\\udf89 Guess what's coming up? We are on the verge of amassing 65,000 subscribers. Upon achieving that milestone, we will be having an Amazon gift card giveaway party!\", \"\\/u\\/techstar2000\", \"<div><p>The title says it all! Be on the look out for this event; you could be one of the twenty (20) lucky winners! <\\/p> <p>Regards, <a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"\\/r\\/Laptopdeals\\\">r\\/Laptopdeals<\\/a> Moderating Team.<\\/p> <\\/div> submitted by <a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/user\\/techstar2000\\\"> \\/u\\/techstar2000 <\\/a> <br \\/><span><a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\\\">[link]<\\/a><\\/span> <span><a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\\\">[comments]<\\/a><\\/span>\", \"the title says it all! be on the look out for this event; you could be one of the twenty (20) lucky winners! regards, r\\/laptopdeals moderating team. submitted by \\/u\\/techstar2000 [link] [comments]\\/u\\/techstar2000\\u23f1\\ufe0f \\u23f3 \\ud83c\\udf89 guess what's coming up? we are on the verge of amassing 65,000 subscribers. upon achieving that milestone, we will be having an amazon gift card giveaway party!https:\\/\\/www.reddit.com\\/r\\/laptopdeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\", \"ebc0cc539e5679038838aad0d9d9151e\", \"4727f6b68615e6246faa85929eefc36a\", 2]:\n\nSQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\\xF0\\x9F\\x8E\\x89 G...' for column `nextcloud`.`oc_news_items`.`title` at row 1","Code":0,"Trace":[{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php","line":169,"function":"convertException","class":"Doctrine\\DBAL\\Driver\\AbstractMySQLDriver","type":"->","args":["An exception occurred while executing 'INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)' with params [true, \"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\", \"t3_lqglv5\", \"ae4638099ccceaabee9bb6828307d5a1\", 1614081022, \"1614094203478929\", \"\\u23f1\\ufe0f \\u23f3 \\ud83c\\udf89 Guess what's coming up? We are on the verge of amassing 65,000 subscribers. Upon achieving that milestone, we will be having an Amazon gift card giveaway party!\", \"\\/u\\/techstar2000\", \"<div><p>The title says it all! Be on the look out for this event; you could be one of the twenty (20) lucky winners! <\\/p> <p>Regards, <a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"\\/r\\/Laptopdeals\\\">r\\/Laptopdeals<\\/a> Moderating Team.<\\/p> <\\/div> submitted by <a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/user\\/techstar2000\\\"> \\/u\\/techstar2000 <\\/a> <br \\/><span><a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\\\">[link]<\\/a><\\/span> <span><a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\\\">[comments]<\\/a><\\/span>\", \"the title says it all! be on the look out for this event; you could be one of the twenty (20) lucky winners! regards, r\\/laptopdeals moderating team. submitted by \\/u\\/techstar2000 [link] [comments]\\/u\\/techstar2000\\u23f1\\ufe0f \\u23f3 \\ud83c\\udf89 guess what's coming up? we are on the verge of amassing 65,000 subscribers. upon achieving that milestone, we will be having an amazon gift card giveaway party!https:\\/\\/www.reddit.com\\/r\\/laptopdeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\", \"ebc0cc539e5679038838aad0d9d9151e\", \"4727f6b68615e6246faa85929eefc36a\", 2]:\n\nSQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\\xF0\\x9F\\x8E\\x89 G...' for column `nextcloud`.`oc_news_items`.`title` at row 1",{"errorInfo":["22007",1366,"Incorrect string value: '\\xF0\\x9F\\x8E\\x89 G...' for column `nextcloud`.`oc_news_items`.`title` at row 1"],"__class__":"Doctrine\\DBAL\\Driver\\PDOException"}]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php","line":149,"function":"wrapException","class":"Doctrine\\DBAL\\DBALException","type":"::","args":[{"__class__":"Doctrine\\DBAL\\Driver\\PDOMySql\\Driver"},{"errorInfo":["22007",1366,"Incorrect string value: '\\xF0\\x9F\\x8E\\x89 G...' for column `nextcloud`.`oc_news_items`.`title` at row 1"],"__class__":"Doctrine\\DBAL\\Driver\\PDOException"},"An exception occurred while executing 'INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)' with params [true, \"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\", \"t3_lqglv5\", \"ae4638099ccceaabee9bb6828307d5a1\", 1614081022, \"1614094203478929\", \"\\u23f1\\ufe0f \\u23f3 \\ud83c\\udf89 Guess what's coming up? We are on the verge of amassing 65,000 subscribers. Upon achieving that milestone, we will be having an Amazon gift card giveaway party!\", \"\\/u\\/techstar2000\", \"<div><p>The title says it all! Be on the look out for this event; you could be one of the twenty (20) lucky winners! <\\/p> <p>Regards, <a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"\\/r\\/Laptopdeals\\\">r\\/Laptopdeals<\\/a> Moderating Team.<\\/p> <\\/div> submitted by <a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/user\\/techstar2000\\\"> \\/u\\/techstar2000 <\\/a> <br \\/><span><a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\\\">[link]<\\/a><\\/span> <span><a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\\\">[comments]<\\/a><\\/span>\", \"the title says it all! be on the look out for this event; you could be one of the twenty (20) lucky winners! regards, r\\/laptopdeals moderating team. submitted by \\/u\\/techstar2000 [link] [comments]\\/u\\/techstar2000\\u23f1\\ufe0f \\u23f3 \\ud83c\\udf89 guess what's coming up? we are on the verge of amassing 65,000 subscribers. upon achieving that milestone, we will be having an amazon gift card giveaway party!https:\\/\\/www.reddit.com\\/r\\/laptopdeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\", \"ebc0cc539e5679038838aad0d9d9151e\", \"4727f6b68615e6246faa85929eefc36a\", 2]:\n\nSQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\\xF0\\x9F\\x8E\\x89 G...' for column `nextcloud`.`oc_news_items`.`title` at row 1"]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php","line":1071,"function":"driverExceptionDuringQuery","class":"Doctrine\\DBAL\\DBALException","type":"::","args":[{"__class__":"Doctrine\\DBAL\\Driver\\PDOMySql\\Driver"},{"errorInfo":["22007",1366,"Incorrect string value: '\\xF0\\x9F\\x8E\\x89 G...' for column `nextcloud`.`oc_news_items`.`title` at row 1"],"__class__":"Doctrine\\DBAL\\Driver\\PDOException"},"INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",[true,"https://www.reddit.com/r/LaptopDeals/comments/lqglv5/guess_whats_coming_up_we_are_on_the_verge_of/","t3_lqglv5","ae4638099ccceaabee9bb6828307d5a1",1614081022,"And 8 more entries, set log level to debug to see all entries"]]},{"file":"/var/www/nextcloud/lib/private/DB/Connection.php","line":222,"function":"executeUpdate","class":"Doctrine\\DBAL\\Connection","type":"->","args":["INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",[true,"https://www.reddit.com/r/LaptopDeals/comments/lqglv5/guess_whats_coming_up_we_are_on_the_verge_of/","t3_lqglv5","ae4638099ccceaabee9bb6828307d5a1",1614081022,"And 8 more entries, set log level to debug to see all entries"],[5,2,2,2,1,"And 8 more entries, set log level to debug to see all entries"]]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php","line":209,"function":"executeUpdate","class":"OC\\DB\\Connection","type":"->","args":["INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(:dcValue1, :dcValue2, :dcValue3, :dcValue4, :dcValue5, :dcValue6, :dcValue7, :dcValue8, :dcValue9, :dcValue10, :dcValue11, :dcValue12, :dcValue13)",{"0":"And 8 more entries, set log level to debug to see all entries","dcValue1":true,"dcValue2":"https://www.reddit.com/r/LaptopDeals/comments/lqglv5/guess_whats_coming_up_we_are_on_the_verge_of/","dcValue3":"t3_lqglv5","dcValue4":"ae4638099ccceaabee9bb6828307d5a1","dcValue5":1614081022},{"0":"And 8 more entries, set log level to debug to see all entries","dcValue1":5,"dcValue2":2,"dcValue3":2,"dcValue4":2,"dcValue5":1}]},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":217,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php","line":135,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/nextcloud/apps/news/lib/Db/NewsMapperV2.php","line":76,"function":"insert","class":"OCP\\AppFramework\\Db\\QBMapper","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/ItemServiceV2.php","line":102,"function":"insert","class":"OCA\\News\\Db\\NewsMapperV2","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/FeedServiceV2.php","line":302,"function":"insertOrUpdate","class":"OCA\\News\\Service\\ItemServiceV2","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/FeedServiceV2.php","line":341,"function":"fetch","class":"OCA\\News\\Service\\FeedServiceV2","type":"->","args":[{"items":[],"id":2,"__class__":"OCA\\News\\Db\\Feed"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/UpdaterService.php","line":55,"function":"fetchAll","class":"OCA\\News\\Service\\FeedServiceV2","type":"->","args":[]},{"file":"/var/www/nextcloud/apps/news/lib/Cron/UpdaterJob.php","line":71,"function":"update","class":"OCA\\News\\Service\\UpdaterService","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/BackgroundJob/Job.php","line":52,"function":"run","class":"OCA\\News\\Cron\\UpdaterJob","type":"->","args":[null]},{"file":"/var/www/nextcloud/lib/private/BackgroundJob/TimedJob.php","line":59,"function":"execute","class":"OC\\BackgroundJob\\Job","type":"->","args":[{"__class__":"OC\\BackgroundJob\\JobList"},{"__class__":"OC\\Log"}]},{"file":"/var/www/nextcloud/cron.php","line":127,"function":"execute","class":"OC\\BackgroundJob\\TimedJob","type":"->","args":[{"__class__":"OC\\BackgroundJob\\JobList"},{"__class__":"OC\\Log"}]}],"File":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractMySQLDriver.php","Line":106,"Previous":{"Exception":"Doctrine\\DBAL\\Driver\\PDOException","Message":"SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\\xF0\\x9F\\x8E\\x89 G...' for column `nextcloud`.`oc_news_items`.`title` at row 1","Code":"22007","Trace":[{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php","line":1062,"function":"execute","class":"Doctrine\\DBAL\\Driver\\PDOStatement","type":"->","args":[]},{"file":"/var/www/nextcloud/lib/private/DB/Connection.php","line":222,"function":"executeUpdate","class":"Doctrine\\DBAL\\Connection","type":"->","args":["INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",[true,"https://www.reddit.com/r/LaptopDeals/comments/lqglv5/guess_whats_coming_up_we_are_on_the_verge_of/","t3_lqglv5","ae4638099ccceaabee9bb6828307d5a1",1614081022,"And 8 more entries, set log level to debug to see all entries"],[5,2,2,2,1,"And 8 more entries, set log level to debug to see all entries"]]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php","line":209,"function":"executeUpdate","class":"OC\\DB\\Connection","type":"->","args":["INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(:dcValue1, :dcValue2, :dcValue3, :dcValue4, :dcValue5, :dcValue6, :dcValue7, :dcValue8, :dcValue9, :dcValue10, :dcValue11, :dcValue12, :dcValue13)",{"0":"And 8 more entries, set log level to debug to see all entries","dcValue1":true,"dcValue2":"https://www.reddit.com/r/LaptopDeals/comments/lqglv5/guess_whats_coming_up_we_are_on_the_verge_of/","dcValue3":"t3_lqglv5","dcValue4":"ae4638099ccceaabee9bb6828307d5a1","dcValue5":1614081022},{"0":"And 8 more entries, set log level to debug to see all entries","dcValue1":5,"dcValue2":2,"dcValue3":2,"dcValue4":2,"dcValue5":1}]},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":217,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php","line":135,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/nextcloud/apps/news/lib/Db/NewsMapperV2.php","line":76,"function":"insert","class":"OCP\\AppFramework\\Db\\QBMapper","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/ItemServiceV2.php","line":102,"function":"insert","class":"OCA\\News\\Db\\NewsMapperV2","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/FeedServiceV2.php","line":302,"function":"insertOrUpdate","class":"OCA\\News\\Service\\ItemServiceV2","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/FeedServiceV2.php","line":341,"function":"fetch","class":"OCA\\News\\Service\\FeedServiceV2","type":"->","args":[{"items":[],"id":2,"__class__":"OCA\\News\\Db\\Feed"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/UpdaterService.php","line":55,"function":"fetchAll","class":"OCA\\News\\Service\\FeedServiceV2","type":"->","args":[]},{"file":"/var/www/nextcloud/apps/news/lib/Cron/UpdaterJob.php","line":71,"function":"update","class":"OCA\\News\\Service\\UpdaterService","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/BackgroundJob/Job.php","line":52,"function":"run","class":"OCA\\News\\Cron\\UpdaterJob","type":"->","args":[null]},{"file":"/var/www/nextcloud/lib/private/BackgroundJob/TimedJob.php","line":59,"function":"execute","class":"OC\\BackgroundJob\\Job","type":"->","args":[{"__class__":"OC\\BackgroundJob\\JobList"},{"__class__":"OC\\Log"}]},{"file":"/var/www/nextcloud/cron.php","line":127,"function":"execute","class":"OC\\BackgroundJob\\TimedJob","type":"->","args":[{"__class__":"OC\\BackgroundJob\\JobList"},{"__class__":"OC\\Log"}]}],"File":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php","Line":129,"Previous":{"Exception":"PDOException","Message":"SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\\xF0\\x9F\\x8E\\x89 G...' for column `nextcloud`.`oc_news_items`.`title` at row 1","Code":"22007","Trace":[{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php","line":127,"function":"execute","class":"PDOStatement","type":"->","args":[null]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php","line":1062,"function":"execute","class":"Doctrine\\DBAL\\Driver\\PDOStatement","type":"->","args":[]},{"file":"/var/www/nextcloud/lib/private/DB/Connection.php","line":222,"function":"executeUpdate","class":"Doctrine\\DBAL\\Connection","type":"->","args":["INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",[true,"https://www.reddit.com/r/LaptopDeals/comments/lqglv5/guess_whats_coming_up_we_are_on_the_verge_of/","t3_lqglv5","ae4638099ccceaabee9bb6828307d5a1",1614081022,"And 8 more entries, set log level to debug to see all entries"],[5,2,2,2,1,"And 8 more entries, set log level to debug to see all entries"]]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php","line":209,"function":"executeUpdate","class":"OC\\DB\\Connection","type":"->","args":["INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(:dcValue1, :dcValue2, :dcValue3, :dcValue4, :dcValue5, :dcValue6, :dcValue7, :dcValue8, :dcValue9, :dcValue10, :dcValue11, :dcValue12, :dcValue13)",{"0":"And 8 more entries, set log level to debug to see all entries","dcValue1":true,"dcValue2":"https://www.reddit.com/r/LaptopDeals/comments/lqglv5/guess_whats_coming_up_we_are_on_the_verge_of/","dcValue3":"t3_lqglv5","dcValue4":"ae4638099ccceaabee9bb6828307d5a1","dcValue5":1614081022},{"0":"And 8 more entries, set log level to debug to see all entries","dcValue1":5,"dcValue2":2,"dcValue3":2,"dcValue4":2,"dcValue5":1}]},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":217,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php","line":135,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/nextcloud/apps/news/lib/Db/NewsMapperV2.php","line":76,"function":"insert","class":"OCP\\AppFramework\\Db\\QBMapper","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/ItemServiceV2.php","line":102,"function":"insert","class":"OCA\\News\\Db\\NewsMapperV2","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/FeedServiceV2.php","line":302,"function":"insertOrUpdate","class":"OCA\\News\\Service\\ItemServiceV2","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/FeedServiceV2.php","line":341,"function":"fetch","class":"OCA\\News\\Service\\FeedServiceV2","type":"->","args":[{"items":[],"id":2,"__class__":"OCA\\News\\Db\\Feed"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/UpdaterService.php","line":55,"function":"fetchAll","class":"OCA\\News\\Service\\FeedServiceV2","type":"->","args":[]},{"file":"/var/www/nextcloud/apps/news/lib/Cron/UpdaterJob.php","line":71,"function":"update","class":"OCA\\News\\Service\\UpdaterService","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/BackgroundJob/Job.php","line":52,"function":"run","class":"OCA\\News\\Cron\\UpdaterJob","type":"->","args":[null]},{"file":"/var/www/nextcloud/lib/private/BackgroundJob/TimedJob.php","line":59,"function":"execute","class":"OC\\BackgroundJob\\Job","type":"->","args":[{"__class__":"OC\\BackgroundJob\\JobList"},{"__class__":"OC\\Log"}]},{"file":"/var/www/nextcloud/cron.php","line":127,"function":"execute","class":"OC\\BackgroundJob\\TimedJob","type":"->","args":[{"__class__":"OC\\BackgroundJob\\JobList"},{"__class__":"OC\\Log"}]}],"File":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php","Line":127}},"CustomMessage":"Error while running background job (class: OCA\\News\\Cron\\UpdaterJob, arguments: )"},"userAgent":"--","version":"20.0.7.1","id":"60352c86ed578"}
@anoymouserver Thanks for letting me know about that, I went through the process but it still threw errors in my log. However, I took one extra step and added a line to my database config file as noted here:
After that I added a test feed back in again and no more errors were generated. I will report back in the event the feeds stop updating again and errors return.
I have the same issue, and indeed I do not have utf8mb4 yet, that's a limitation of yunohost package currently. Would it be possible to at least catch the error so that only the problematic feed do not update? Also having a list of feeds with errors somewhere would be useful.
Explain the Problem
Since I updated NextCloud and also the News app most of my feeds aren't updated anymore. This seems to be caused by this feed: http://www.windytan.com/feeds/posts/default It is a valid Atom 1.0 feed. feed-io failed for me, because Firefox denied the "consume" request with "CORS Missing Allow Origin". But i think fetching is fine, as an exception is logged in nextcloud.log only when inserting the data to the database. This probably leads to all following feeds not being updated.
This started with my update of NextCloud and News. This is a grep on "read access" in the log showing this clearly:
System Information
Contents of nextcloud/data/nextcloud.log
```json {"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:20+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : http://www.windytan.com/feeds/posts/default into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.5.2"} {"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:20+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"start reading http://www.windytan.com/feeds/posts/default","userAgent":"--","version":"20.0.5.2"} {"reqId":"1AscWZy8T7CHNOyvh5Mo","level":1,"time":"2021-01-23T22:34:20+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"no 'modifiedSince' parameter given, setting it to 01/01/1970","userAgent":"--","version":"20.0.5.2"} {"reqId":"1AscWZy8T7CHNOyvh5Mo","level":1,"time":"2021-01-23T22:34:20+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"hitting http://www.windytan.com/feeds/posts/default","userAgent":"--","version":"20.0.5.2"} {"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"response ok, now turning it into a document","userAgent":"--","version":"20.0.5.2"} {"reqId":"1AscWZy8T7CHNOyvh5Mo","level":1,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"the stream is modified, parsing it","userAgent":"--","version":"20.0.5.2"} {"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"accurate parser : FeedIo\\Parser\\XmlParser","userAgent":"--","version":"20.0.5.2"} {"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"Feed http://www.windytan.com/feeds/posts/default was modified since last fetch. #25 items","userAgent":"--","version":"20.0.5.2"} {"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"Added item Plotting patterns in music with a fantasy record player for feed absorptions publishdate: 1608189512","userAgent":"--","version":"20.0.5.2"} {"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"Added item Capturing PAL video with an SDR (and a few dead-ends) for feed absorptions publishdate: 1589710223","userAgent":"--","version":"20.0.5.2"} {"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"Added item Beeps and melodies in two-way radio for feed absorptions publishdate: 1566587018","userAgent":"--","version":"20.0.5.2"} {"reqId":"1AscWZy8T7CHNOyvh5Mo","level":3,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"core","method":"","url":"--","message":{"Exception":"Doctrine\\DBAL\\Exception\\DriverException","Message":"An exception occurred while executing 'UPDATE `oc_news_items` SET `unread` = ?, `url` = ?, `guid` = ?, `guid_hash` = ?, `pub_date` = ?, `last_modified` = ?, `title` = ?, `author` = ?, `body` = ?, `search_index` = ?, `fingerprint` = ?, `content_hash` = ?, `feed_id` = ? WHERE `id` = ?' with params [false, \"http:\\/\\/www.windytan.com\\/2014\\/10\\/visualizing-hex-bytes-with-unicode-emoji.html\", \"tag:blogger.com,1999:blog-5096278891763426276.post-7039745451118618656\", \"a47b16a321e3f78a36d0ab89e53e505b\", 1519239491, \"1611441261931408\", \"Visualizing hex dumps with Unicode emoji\", \"Oona R\\u00e4is\\u00e4nen\", \"Memorizing SSH public key fingerprints can be difficult; they're just long random numbers displayed in base 16. There are some terminal-friendly solutions, like OpenSSH's randomart. But because I use a Unicode terminal, I like to map the individual bytes into characters in the Miscellaneous Symbols and Pictographs<\\/a> block.<\\/p>
This Perl script does just that:<\\/p>