nextcloud / news

:newspaper: RSS/Atom feed reader
https://apps.nextcloud.com/apps/news
GNU Affero General Public License v3.0
866 stars 186 forks source link

After update: Exception during INSERT: Invalid datetime format: 1366 Incorrect string value: '\\xF0\\x9F\\x8C\\x80 #1165

Closed Jonny007-MKD closed 3 years ago

Jonny007-MKD commented 3 years ago

Explain the Problem

Since I updated NextCloud and also the News app most of my feeds aren't updated anymore. This seems to be caused by this feed: http://www.windytan.com/feeds/posts/default It is a valid Atom 1.0 feed. feed-io failed for me, because Firefox denied the "consume" request with "CORS Missing Allow Origin". But i think fetching is fine, as an exception is logged in nextcloud.log only when inserting the data to the database. This probably leads to all following feeds not being updated.

This started with my update of NextCloud and News. This is a grep on "read access" in the log showing this clearly:

... even more feeds in this cron call
{"reqId":"iYCN1BkNdPgjQ86n5CSj","level":0,"time":"2021-01-23T21:34:48+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : https://foonathan.net/post/index.xml into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.4.0"}
{"reqId":"iYCN1BkNdPgjQ86n5CSj","level":0,"time":"2021-01-23T21:34:48+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : https://github.com/kimai/kimai/releases.atom into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.4.0"}
{"reqId":"iYCN1BkNdPgjQ86n5CSj","level":0,"time":"2021-01-23T21:34:48+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : https://blog.linuxmint.com/?feed=rss2 into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.4.0"}
{"reqId":"iYCN1BkNdPgjQ86n5CSj","level":0,"time":"2021-01-23T21:34:48+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : https://blog.gitea.io/index.xml into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.4.0"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:20+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : http://www.tagesschau.de/xml/rss2 into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:20+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : http://www.windytan.com/feeds/posts/default into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.5.2"}
{"reqId":"KGIXbcHcmeEDp796Z9Mf","level":0,"time":"2021-01-23T23:34:27+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : http://www.tagesschau.de/xml/rss2 into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.5.2"}
{"reqId":"KGIXbcHcmeEDp796Z9Mf","level":0,"time":"2021-01-23T23:34:27+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : http://www.windytan.com/feeds/posts/default into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.5.2"}

System Information

Contents of nextcloud/data/nextcloud.log ```json {"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:20+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : http://www.windytan.com/feeds/posts/default into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.5.2"} {"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:20+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"start reading http://www.windytan.com/feeds/posts/default","userAgent":"--","version":"20.0.5.2"} {"reqId":"1AscWZy8T7CHNOyvh5Mo","level":1,"time":"2021-01-23T22:34:20+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"no 'modifiedSince' parameter given, setting it to 01/01/1970","userAgent":"--","version":"20.0.5.2"} {"reqId":"1AscWZy8T7CHNOyvh5Mo","level":1,"time":"2021-01-23T22:34:20+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"hitting http://www.windytan.com/feeds/posts/default","userAgent":"--","version":"20.0.5.2"} {"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"response ok, now turning it into a document","userAgent":"--","version":"20.0.5.2"} {"reqId":"1AscWZy8T7CHNOyvh5Mo","level":1,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"the stream is modified, parsing it","userAgent":"--","version":"20.0.5.2"} {"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"accurate parser : FeedIo\\Parser\\XmlParser","userAgent":"--","version":"20.0.5.2"} {"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"Feed http://www.windytan.com/feeds/posts/default was modified since last fetch. #25 items","userAgent":"--","version":"20.0.5.2"} {"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"Added item Plotting patterns in music with a fantasy record player for feed absorptions publishdate: 1608189512","userAgent":"--","version":"20.0.5.2"} {"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"Added item Capturing PAL video with an SDR (and a few dead-ends) for feed absorptions publishdate: 1589710223","userAgent":"--","version":"20.0.5.2"} {"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"Added item Beeps and melodies in two-way radio for feed absorptions publishdate: 1566587018","userAgent":"--","version":"20.0.5.2"} {"reqId":"1AscWZy8T7CHNOyvh5Mo","level":3,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"core","method":"","url":"--","message":{"Exception":"Doctrine\\DBAL\\Exception\\DriverException","Message":"An exception occurred while executing 'UPDATE `oc_news_items` SET `unread` = ?, `url` = ?, `guid` = ?, `guid_hash` = ?, `pub_date` = ?, `last_modified` = ?, `title` = ?, `author` = ?, `body` = ?, `search_index` = ?, `fingerprint` = ?, `content_hash` = ?, `feed_id` = ? WHERE `id` = ?' with params [false, \"http:\\/\\/www.windytan.com\\/2014\\/10\\/visualizing-hex-bytes-with-unicode-emoji.html\", \"tag:blogger.com,1999:blog-5096278891763426276.post-7039745451118618656\", \"a47b16a321e3f78a36d0ab89e53e505b\", 1519239491, \"1611441261931408\", \"Visualizing hex dumps with Unicode emoji\", \"Oona R\\u00e4is\\u00e4nen\", \"

Memorizing SSH public key fingerprints can be difficult; they're just long random numbers displayed in base 16. There are some terminal-friendly solutions, like OpenSSH's randomart. But because I use a Unicode terminal, I like to map the individual bytes into characters in the Miscellaneous Symbols and Pictographs<\\/a> block.<\\/p>

This Perl script does just that:<\\/p>


@emoji = qw( \\ud83c\\udf00 \\ud83c\\udf02 \\ud83c\\udf05 \\ud83c\\udf08 \\ud83c\\udf19 \\ud83c\\udf1e \\ud83c\\udf1f \\ud83c\\udf20 \\ud83c\\udf30 \\ud83c\\udf31 \\ud83c\\udf32 \\ud83c\\udf33 \\ud83c\\udf34 \\ud83c\\udf35 \\ud83c\\udf37 \\ud83c\\udf38
\\ud83c\\udf39 \\ud83c\\udf3a \\ud83c\\udf3b \\ud83c\\udf3c \\ud83c\\udf3d \\ud83c\\udf3e \\ud83c\\udf3f \\ud83c\\udf40 \\ud83c\\udf41 \\ud83c\\udf42 \\ud83c\\udf43 \\ud83c\\udf44 \\ud83c\\udf45 \\ud83c\\udf46 \\ud83c\\udf47 \\ud83c\\udf48
\\ud83c\\udf49 \\ud83c\\udf4a \\ud83c\\udf4b \\ud83c\\udf4c \\ud83c\\udf4d \\ud83c\\udf4e \\ud83c\\udf4f \\ud83c\\udf50 \\ud83c\\udf51 \\ud83c\\udf52 \\ud83c\\udf53 \\ud83c\\udf54 \\ud83c\\udf55 \\ud83c\\udf56 \\ud83c\\udf57 \\ud83c\\udf58
\\ud83c\\udf5c \\ud83c\\udf5d \\ud83c\\udf5e \\ud83c\\udf5f \\ud83c\\udf60 \\ud83c\\udf61 \\ud83c\\udf62 \\ud83c\\udf63 \\ud83c\\udf64 \\ud83c\\udf65 \\ud83c\\udf66 \\ud83c\\udf67 \\ud83c\\udf68 \\ud83c\\udf69 \\ud83c\\udf6a \\ud83c\\udf6b
\\ud83c\\udf6c \\ud83c\\udf6d \\ud83c\\udf6e \\ud83c\\udf6f \\ud83c\\udf70 \\ud83c\\udf71 \\ud83c\\udf72 \\ud83c\\udf73 \\ud83c\\udf74 \\ud83c\\udf75 \\ud83c\\udf76 \\ud83c\\udf77 \\ud83c\\udf78 \\ud83c\\udf79 \\ud83c\\udf7a \\ud83c\\udf7b
\\ud83c\\udf7c \\ud83c\\udf80 \\ud83c\\udf81 \\ud83c\\udf82 \\ud83c\\udf83 \\ud83c\\udf84 \\ud83c\\udf85 \\ud83c\\udf88 \\ud83c\\udf89 \\ud83c\\udf8a \\ud83c\\udf8b \\ud83c\\udf8c \\ud83c\\udf8d \\ud83c\\udf8e \\ud83c\\udf8f \\ud83c\\udf92
\\ud83c\\udf93 \\ud83c\\udfa0 \\ud83c\\udfa1 \\ud83c\\udfa2 \\ud83c\\udfa3 \\ud83c\\udfa4 \\ud83c\\udfa5 \\ud83c\\udfa6 \\ud83c\\udfa7 \\ud83c\\udfa8 \\ud83c\\udfa9 \\ud83c\\udfaa \\ud83c\\udfab \\ud83c\\udfac \\ud83c\\udfad \\ud83c\\udfae
\\ud83c\\udfaf \\ud83c\\udfb0 \\ud83c\\udfb1 \\ud83c\\udfb2 \\ud83c\\udfb3 \\ud83c\\udfb4 \\ud83c\\udfb5 \\ud83c\\udfb7 \\ud83c\\udfb8 \\ud83c\\udfb9 \\ud83c\\udfba \\ud83c\\udfbb \\ud83c\\udfbd \\ud83c\\udfbe \\ud83c\\udfbf \\ud83c\\udfc0
\\ud83c\\udfc1 \\ud83c\\udfc2 \\ud83c\\udfc3 \\ud83c\\udfc4 \\ud83c\\udfc6 \\ud83c\\udfc7 \\ud83c\\udfc8 \\ud83c\\udfc9 \\ud83c\\udfca \\ud83d\\udc00 \\ud83d\\udc01 \\ud83d\\udc02 \\ud83d\\udc03 \\ud83d\\udc04 \\ud83d\\udc05 \\ud83d\\udc06
\\ud83d\\udc07 \\ud83d\\udc08 \\ud83d\\udc09 \\ud83d\\udc0a \\ud83d\\udc0b \\ud83d\\udc0c \\ud83d\\udc0d \\ud83d\\udc0e \\ud83d\\udc0f \\ud83d\\udc10 \\ud83d\\udc11 \\ud83d\\udc12 \\ud83d\\udc13 \\ud83d\\udc14 \\ud83d\\udc15 \\ud83d\\udc16
\\ud83d\\udc17 \\ud83d\\udc18 \\ud83d\\udc19 \\ud83d\\udc1a \\ud83d\\udc1b \\ud83d\\udc1c \\ud83d\\udc1d \\ud83d\\udc1e \\ud83d\\udc1f \\ud83d\\udc20 \\ud83d\\udc21 \\ud83d\\udc22 \\ud83d\\udc23 \\ud83d\\udc24 \\ud83d\\udc25 \\ud83d\\udc26
\\ud83d\\udc27 \\ud83d\\udc28 \\ud83d\\udc29 \\ud83d\\udc2a \\ud83d\\udc2b \\ud83d\\udc2c \\ud83d\\udc2d \\ud83d\\udc2e \\ud83d\\udc2f \\ud83d\\udc30 \\ud83d\\udc31 \\ud83d\\udc32 \\ud83d\\udc33 \\ud83d\\udc34 \\ud83d\\udc35 \\ud83d\\udc36
\\ud83d\\udc37 \\ud83d\\udc38 \\ud83d\\udc39 \\ud83d\\udc3a \\ud83d\\udc3b \\ud83d\\udc3c \\ud83d\\udc3d \\ud83d\\udc3e \\ud83d\\udc40 \\ud83d\\udc42 \\ud83d\\udc43 \\ud83d\\udc44 \\ud83d\\udc45 \\ud83d\\udc46 \\ud83d\\udc47 \\ud83d\\udc48
\\ud83d\\udc49 \\ud83d\\udc4a \\ud83d\\udc4b \\ud83d\\udc4c \\ud83d\\udc4d \\ud83d\\udc4e \\ud83d\\udc4f \\ud83d\\udc50 \\ud83d\\udc51 \\ud83d\\udc52 \\ud83d\\udc53 \\ud83d\\udc54 \\ud83d\\udc55 \\ud83d\\udc56 \\ud83d\\udc57 \\ud83d\\udc58
\\ud83d\\udc59 \\ud83d\\udc5a \\ud83d\\udc5b \\ud83d\\udc5c \\ud83d\\udc5d \\ud83d\\udc5e \\ud83d\\udc5f \\ud83d\\udc60 \\ud83d\\udc61 \\ud83d\\udc62 \\ud83d\\udc63 \\ud83d\\udc64 \\ud83d\\udc65 \\ud83d\\udc66 \\ud83d\\udc67 \\ud83d\\udc68
\\ud83d\\udc69 \\ud83d\\udc6a \\ud83d\\udc6e \\ud83d\\udc6f \\ud83d\\udc7a \\ud83d\\udc7b \\ud83d\\udc7c \\ud83d\\udc7d \\ud83d\\udc7e \\ud83d\\udc7f \\ud83d\\udc80 \\ud83d\\udc81 \\ud83d\\udc82 \\ud83d\\udc83 \\ud83d\\udc84 \\ud83d\\udc85 );

while (<>) {
if (\\/[a-f0-9:]+:[a-f0-9:]+\\/) {
($b, $m, $a) = ($`, $&, $');
print $b.join(\\\" \\\", map { $emoji[$_] } map hex, split \\/:\\/, $m).\\\" \\\".$a;
}
}<\\/code><\\/pre>

What's happening here? First we create a 256-element array containing a hand-picked collection of emoji. Naturally, they're all assigned an index from 0x00<\\/span> to 0xff<\\/span>. Then we'll loop through standard input and look for lines containing colon-separated hex bytes. Each hex value is replaced with an emoji from the array.<\\/p>

Here's the output:<\\/p>

\\\"[Image:<\\/div>

The script could easily be extended to support output from other hex-formatted sources as well, such as xxd:<\\/p>

\\\"[Image:<\\/div>

Some additional methods for visualizing hex dumps and key fingerprints, from the comments section:<\\/p>

Grotax commented 3 years ago

I reported the feed-io issue https://github.com/alexdebril/feed-io/issues/325

And you are right, fetching the feed is no problem I tried it on cli with feed-io. The issue happens when news tries to insert the data into the DB, I have to say though that I don't understand the error message.

SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\\xF0\\x9F\\x8C\\x80  ...' for column `newown_nordlichter_brv_de`.`oc_news_items`.`body` at row 1",

The full error, isolated:

```json { "reqId": "1AscWZy8T7CHNOyvh5Mo", "level": 3, "time": "2021-01-23T22:34:21+00:00", "remoteAddr": "", "user": "--", "app": "core", "method": "", "url": "--", "message": { "Exception": "Doctrine\\DBAL\\Exception\\DriverException", "Message": "An exception occurred while executing 'UPDATE `oc_news_items` SET `unread` = ?, `url` = ?, `guid` = ?, `guid_hash` = ?, `pub_date` = ?, `last_modified` = ?, `title` = ?, `author` = ?, `body` = ?, `search_index` = ?, `fingerprint` = ?, `content_hash` = ?, `feed_id` = ? WHERE `id` = ?' with params [false, \"http:\\/\\/www.windytan.com\\/2014\\/10\\/visualizing-hex-bytes-with-unicode-emoji.html\", \"tag:blogger.com,1999:blog-5096278891763426276.post-7039745451118618656\", \"a47b16a321e3f78a36d0ab89e53e505b\", 1519239491, \"1611441261931408\", \"Visualizing hex dumps with Unicode emoji\", \"Oona R\\u00e4is\\u00e4nen\", \"

Memorizing SSH public key fingerprints can be difficult; they're just long random numbers displayed in base 16. There are some terminal-friendly solutions, like OpenSSH's randomart. But because I use a Unicode terminal, I like to map the individual bytes into characters in the Miscellaneous Symbols and Pictographs<\\/a> block.<\\/p>

This Perl script does just that:<\\/p>


@emoji = qw( \\ud83c\\udf00 \\ud83c\\udf02 \\ud83c\\udf05 \\ud83c\\udf08 \\ud83c\\udf19 \\ud83c\\udf1e \\ud83c\\udf1f \\ud83c\\udf20 \\ud83c\\udf30 \\ud83c\\udf31 \\ud83c\\udf32 \\ud83c\\udf33 \\ud83c\\udf34 \\ud83c\\udf35 \\ud83c\\udf37 \\ud83c\\udf38
\\ud83c\\udf39 \\ud83c\\udf3a \\ud83c\\udf3b \\ud83c\\udf3c \\ud83c\\udf3d \\ud83c\\udf3e \\ud83c\\udf3f \\ud83c\\udf40 \\ud83c\\udf41 \\ud83c\\udf42 \\ud83c\\udf43 \\ud83c\\udf44 \\ud83c\\udf45 \\ud83c\\udf46 \\ud83c\\udf47 \\ud83c\\udf48
\\ud83c\\udf49 \\ud83c\\udf4a \\ud83c\\udf4b \\ud83c\\udf4c \\ud83c\\udf4d \\ud83c\\udf4e \\ud83c\\udf4f \\ud83c\\udf50 \\ud83c\\udf51 \\ud83c\\udf52 \\ud83c\\udf53 \\ud83c\\udf54 \\ud83c\\udf55 \\ud83c\\udf56 \\ud83c\\udf57 \\ud83c\\udf58
\\ud83c\\udf5c \\ud83c\\udf5d \\ud83c\\udf5e \\ud83c\\udf5f \\ud83c\\udf60 \\ud83c\\udf61 \\ud83c\\udf62 \\ud83c\\udf63 \\ud83c\\udf64 \\ud83c\\udf65 \\ud83c\\udf66 \\ud83c\\udf67 \\ud83c\\udf68 \\ud83c\\udf69 \\ud83c\\udf6a \\ud83c\\udf6b
\\ud83c\\udf6c \\ud83c\\udf6d \\ud83c\\udf6e \\ud83c\\udf6f \\ud83c\\udf70 \\ud83c\\udf71 \\ud83c\\udf72 \\ud83c\\udf73 \\ud83c\\udf74 \\ud83c\\udf75 \\ud83c\\udf76 \\ud83c\\udf77 \\ud83c\\udf78 \\ud83c\\udf79 \\ud83c\\udf7a \\ud83c\\udf7b
\\ud83c\\udf7c \\ud83c\\udf80 \\ud83c\\udf81 \\ud83c\\udf82 \\ud83c\\udf83 \\ud83c\\udf84 \\ud83c\\udf85 \\ud83c\\udf88 \\ud83c\\udf89 \\ud83c\\udf8a \\ud83c\\udf8b \\ud83c\\udf8c \\ud83c\\udf8d \\ud83c\\udf8e \\ud83c\\udf8f \\ud83c\\udf92
\\ud83c\\udf93 \\ud83c\\udfa0 \\ud83c\\udfa1 \\ud83c\\udfa2 \\ud83c\\udfa3 \\ud83c\\udfa4 \\ud83c\\udfa5 \\ud83c\\udfa6 \\ud83c\\udfa7 \\ud83c\\udfa8 \\ud83c\\udfa9 \\ud83c\\udfaa \\ud83c\\udfab \\ud83c\\udfac \\ud83c\\udfad \\ud83c\\udfae
\\ud83c\\udfaf \\ud83c\\udfb0 \\ud83c\\udfb1 \\ud83c\\udfb2 \\ud83c\\udfb3 \\ud83c\\udfb4 \\ud83c\\udfb5 \\ud83c\\udfb7 \\ud83c\\udfb8 \\ud83c\\udfb9 \\ud83c\\udfba \\ud83c\\udfbb \\ud83c\\udfbd \\ud83c\\udfbe \\ud83c\\udfbf \\ud83c\\udfc0
\\ud83c\\udfc1 \\ud83c\\udfc2 \\ud83c\\udfc3 \\ud83c\\udfc4 \\ud83c\\udfc6 \\ud83c\\udfc7 \\ud83c\\udfc8 \\ud83c\\udfc9 \\ud83c\\udfca \\ud83d\\udc00 \\ud83d\\udc01 \\ud83d\\udc02 \\ud83d\\udc03 \\ud83d\\udc04 \\ud83d\\udc05 \\ud83d\\udc06
\\ud83d\\udc07 \\ud83d\\udc08 \\ud83d\\udc09 \\ud83d\\udc0a \\ud83d\\udc0b \\ud83d\\udc0c \\ud83d\\udc0d \\ud83d\\udc0e \\ud83d\\udc0f \\ud83d\\udc10 \\ud83d\\udc11 \\ud83d\\udc12 \\ud83d\\udc13 \\ud83d\\udc14 \\ud83d\\udc15 \\ud83d\\udc16
\\ud83d\\udc17 \\ud83d\\udc18 \\ud83d\\udc19 \\ud83d\\udc1a \\ud83d\\udc1b \\ud83d\\udc1c \\ud83d\\udc1d \\ud83d\\udc1e \\ud83d\\udc1f \\ud83d\\udc20 \\ud83d\\udc21 \\ud83d\\udc22 \\ud83d\\udc23 \\ud83d\\udc24 \\ud83d\\udc25 \\ud83d\\udc26
\\ud83d\\udc27 \\ud83d\\udc28 \\ud83d\\udc29 \\ud83d\\udc2a \\ud83d\\udc2b \\ud83d\\udc2c \\ud83d\\udc2d \\ud83d\\udc2e \\ud83d\\udc2f \\ud83d\\udc30 \\ud83d\\udc31 \\ud83d\\udc32 \\ud83d\\udc33 \\ud83d\\udc34 \\ud83d\\udc35 \\ud83d\\udc36
\\ud83d\\udc37 \\ud83d\\udc38 \\ud83d\\udc39 \\ud83d\\udc3a \\ud83d\\udc3b \\ud83d\\udc3c \\ud83d\\udc3d \\ud83d\\udc3e \\ud83d\\udc40 \\ud83d\\udc42 \\ud83d\\udc43 \\ud83d\\udc44 \\ud83d\\udc45 \\ud83d\\udc46 \\ud83d\\udc47 \\ud83d\\udc48
\\ud83d\\udc49 \\ud83d\\udc4a \\ud83d\\udc4b \\ud83d\\udc4c \\ud83d\\udc4d \\ud83d\\udc4e \\ud83d\\udc4f \\ud83d\\udc50 \\ud83d\\udc51 \\ud83d\\udc52 \\ud83d\\udc53 \\ud83d\\udc54 \\ud83d\\udc55 \\ud83d\\udc56 \\ud83d\\udc57 \\ud83d\\udc58
\\ud83d\\udc59 \\ud83d\\udc5a \\ud83d\\udc5b \\ud83d\\udc5c \\ud83d\\udc5d \\ud83d\\udc5e \\ud83d\\udc5f \\ud83d\\udc60 \\ud83d\\udc61 \\ud83d\\udc62 \\ud83d\\udc63 \\ud83d\\udc64 \\ud83d\\udc65 \\ud83d\\udc66 \\ud83d\\udc67 \\ud83d\\udc68
\\ud83d\\udc69 \\ud83d\\udc6a \\ud83d\\udc6e \\ud83d\\udc6f \\ud83d\\udc7a \\ud83d\\udc7b \\ud83d\\udc7c \\ud83d\\udc7d \\ud83d\\udc7e \\ud83d\\udc7f \\ud83d\\udc80 \\ud83d\\udc81 \\ud83d\\udc82 \\ud83d\\udc83 \\ud83d\\udc84 \\ud83d\\udc85 );

while (<>) {
if (\\/[a-f0-9:]+:[a-f0-9:]+\\/) {
($b, $m, $a) = ($`, $&, $');
print $b.join(\\\" \\\", map { $emoji[$_] } map hex, split \\/:\\/, $m).\\\" \\\".$a;
}
}<\\/code><\\/pre>

What's happening here? First we create a 256-element array containing a hand-picked collection of emoji. Naturally, they're all assigned an index from 0x00<\\/span> to 0xff<\\/span>. Then we'll loop through standard input and look for lines containing colon-separated hex bytes. Each hex value is replaced with an emoji from the array.<\\/p>

Here's the output:<\\/p>

\\\"[Image:<\\/div>

The script could easily be extended to support output from other hex-formatted sources as well, such as xxd:<\\/p>

\\\"[Image:<\\/div>

Some additional methods for visualizing hex dumps and key fingerprints, from the comments section:<\\/p>

Jonny007-MKD commented 3 years ago

Probably the source of the error lies in the body of the post "Animated line drawings with OpenCV", as the previous posts were added (according to the nextcloud.log). For me it seems to be valid UTF-8, I could not find any chars apart [-a-zA-Z0-9<>,.;:-_"\/\\+#\'&(){}*%$!|] in it. Perhaps something is interpreted incorrectly due to some \s?

anoymouserver commented 3 years ago

It happens in the Visualizing hex dumps with Unicode emoji article.

The escape sequences are exactly the ones of the first emoji:

🌀                🌂                ...
\xF0\x9F\x8C\x80  \xF0\x9F\x8C\x82
Jonny007-MKD commented 3 years ago

As a temporary workaround I set the prevent_update flag in the oc_news_feeds to 1 for this feed. Now all the other feeds are successfully fetched again.

Jdogzz commented 3 years ago

I can add a couple extra datapoints with these two rss feeds yielding the invalid datetime format error: https://www.reddit.com/r/LaptopDeals/.rss https://www.reddit.com/r/PCDeals/.rss

They give, for example, this error in my logs:

{"reqId":"uatNiR4c0l1uWj2bT5fY","level":3,"time":"2021-02-23T15:30:03+00:00","remoteAddr":"","user":"--","app":"core","method":"","url":"--","message":{"Exception":"Doctrine\\DBAL\\Exception\\DriverException","Message":"An exception occurred while executing 'INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)' with params [true, \"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\", \"t3_lqglv5\", \"ae4638099ccceaabee9bb6828307d5a1\", 1614081022, \"1614094203478929\", \"\\u23f1\\ufe0f \\u23f3 \\ud83c\\udf89 Guess what's coming up? We are on the verge of amassing 65,000 subscribers. Upon achieving that milestone, we will be having an Amazon gift card giveaway party!\", \"\\/u\\/techstar2000\", \"<div><p>The title says it all! Be on the look out for this event; you could be one of the twenty (20) lucky winners! <\\/p> <p>Regards, <a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"\\/r\\/Laptopdeals\\\">r\\/Laptopdeals<\\/a> Moderating Team.<\\/p> <\\/div>   submitted by   <a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/user\\/techstar2000\\\"> \\/u\\/techstar2000 <\\/a> <br \\/><span><a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\\\">[link]<\\/a><\\/span>   <span><a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\\\">[comments]<\\/a><\\/span>\", \"the title says it all! be on the look out for this event; you could be one of the twenty (20) lucky winners!  regards, r\\/laptopdeals moderating team.    submitted by    \\/u\\/techstar2000   [link]   [comments]\\/u\\/techstar2000\\u23f1\\ufe0f \\u23f3 \\ud83c\\udf89 guess what's coming up? we are on the verge of amassing 65,000 subscribers. upon achieving that milestone, we will be having an amazon gift card giveaway party!https:\\/\\/www.reddit.com\\/r\\/laptopdeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\", \"ebc0cc539e5679038838aad0d9d9151e\", \"4727f6b68615e6246faa85929eefc36a\", 2]:\n\nSQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\\xF0\\x9F\\x8E\\x89 G...' for column `nextcloud`.`oc_news_items`.`title` at row 1","Code":0,"Trace":[{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php","line":169,"function":"convertException","class":"Doctrine\\DBAL\\Driver\\AbstractMySQLDriver","type":"->","args":["An exception occurred while executing 'INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)' with params [true, \"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\", \"t3_lqglv5\", \"ae4638099ccceaabee9bb6828307d5a1\", 1614081022, \"1614094203478929\", \"\\u23f1\\ufe0f \\u23f3 \\ud83c\\udf89 Guess what's coming up? We are on the verge of amassing 65,000 subscribers. Upon achieving that milestone, we will be having an Amazon gift card giveaway party!\", \"\\/u\\/techstar2000\", \"<div><p>The title says it all! Be on the look out for this event; you could be one of the twenty (20) lucky winners! <\\/p> <p>Regards, <a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"\\/r\\/Laptopdeals\\\">r\\/Laptopdeals<\\/a> Moderating Team.<\\/p> <\\/div>   submitted by   <a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/user\\/techstar2000\\\"> \\/u\\/techstar2000 <\\/a> <br \\/><span><a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\\\">[link]<\\/a><\\/span>   <span><a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\\\">[comments]<\\/a><\\/span>\", \"the title says it all! be on the look out for this event; you could be one of the twenty (20) lucky winners!  regards, r\\/laptopdeals moderating team.    submitted by    \\/u\\/techstar2000   [link]   [comments]\\/u\\/techstar2000\\u23f1\\ufe0f \\u23f3 \\ud83c\\udf89 guess what's coming up? we are on the verge of amassing 65,000 subscribers. upon achieving that milestone, we will be having an amazon gift card giveaway party!https:\\/\\/www.reddit.com\\/r\\/laptopdeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\", \"ebc0cc539e5679038838aad0d9d9151e\", \"4727f6b68615e6246faa85929eefc36a\", 2]:\n\nSQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\\xF0\\x9F\\x8E\\x89 G...' for column `nextcloud`.`oc_news_items`.`title` at row 1",{"errorInfo":["22007",1366,"Incorrect string value: '\\xF0\\x9F\\x8E\\x89 G...' for column `nextcloud`.`oc_news_items`.`title` at row 1"],"__class__":"Doctrine\\DBAL\\Driver\\PDOException"}]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php","line":149,"function":"wrapException","class":"Doctrine\\DBAL\\DBALException","type":"::","args":[{"__class__":"Doctrine\\DBAL\\Driver\\PDOMySql\\Driver"},{"errorInfo":["22007",1366,"Incorrect string value: '\\xF0\\x9F\\x8E\\x89 G...' for column `nextcloud`.`oc_news_items`.`title` at row 1"],"__class__":"Doctrine\\DBAL\\Driver\\PDOException"},"An exception occurred while executing 'INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)' with params [true, \"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\", \"t3_lqglv5\", \"ae4638099ccceaabee9bb6828307d5a1\", 1614081022, \"1614094203478929\", \"\\u23f1\\ufe0f \\u23f3 \\ud83c\\udf89 Guess what's coming up? We are on the verge of amassing 65,000 subscribers. Upon achieving that milestone, we will be having an Amazon gift card giveaway party!\", \"\\/u\\/techstar2000\", \"<div><p>The title says it all! Be on the look out for this event; you could be one of the twenty (20) lucky winners! <\\/p> <p>Regards, <a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"\\/r\\/Laptopdeals\\\">r\\/Laptopdeals<\\/a> Moderating Team.<\\/p> <\\/div>   submitted by   <a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/user\\/techstar2000\\\"> \\/u\\/techstar2000 <\\/a> <br \\/><span><a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\\\">[link]<\\/a><\\/span>   <span><a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\\\">[comments]<\\/a><\\/span>\", \"the title says it all! be on the look out for this event; you could be one of the twenty (20) lucky winners!  regards, r\\/laptopdeals moderating team.    submitted by    \\/u\\/techstar2000   [link]   [comments]\\/u\\/techstar2000\\u23f1\\ufe0f \\u23f3 \\ud83c\\udf89 guess what's coming up? we are on the verge of amassing 65,000 subscribers. upon achieving that milestone, we will be having an amazon gift card giveaway party!https:\\/\\/www.reddit.com\\/r\\/laptopdeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\", \"ebc0cc539e5679038838aad0d9d9151e\", \"4727f6b68615e6246faa85929eefc36a\", 2]:\n\nSQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\\xF0\\x9F\\x8E\\x89 G...' for column `nextcloud`.`oc_news_items`.`title` at row 1"]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php","line":1071,"function":"driverExceptionDuringQuery","class":"Doctrine\\DBAL\\DBALException","type":"::","args":[{"__class__":"Doctrine\\DBAL\\Driver\\PDOMySql\\Driver"},{"errorInfo":["22007",1366,"Incorrect string value: '\\xF0\\x9F\\x8E\\x89 G...' for column `nextcloud`.`oc_news_items`.`title` at row 1"],"__class__":"Doctrine\\DBAL\\Driver\\PDOException"},"INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",[true,"https://www.reddit.com/r/LaptopDeals/comments/lqglv5/guess_whats_coming_up_we_are_on_the_verge_of/","t3_lqglv5","ae4638099ccceaabee9bb6828307d5a1",1614081022,"And 8 more entries, set log level to debug to see all entries"]]},{"file":"/var/www/nextcloud/lib/private/DB/Connection.php","line":222,"function":"executeUpdate","class":"Doctrine\\DBAL\\Connection","type":"->","args":["INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",[true,"https://www.reddit.com/r/LaptopDeals/comments/lqglv5/guess_whats_coming_up_we_are_on_the_verge_of/","t3_lqglv5","ae4638099ccceaabee9bb6828307d5a1",1614081022,"And 8 more entries, set log level to debug to see all entries"],[5,2,2,2,1,"And 8 more entries, set log level to debug to see all entries"]]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php","line":209,"function":"executeUpdate","class":"OC\\DB\\Connection","type":"->","args":["INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(:dcValue1, :dcValue2, :dcValue3, :dcValue4, :dcValue5, :dcValue6, :dcValue7, :dcValue8, :dcValue9, :dcValue10, :dcValue11, :dcValue12, :dcValue13)",{"0":"And 8 more entries, set log level to debug to see all entries","dcValue1":true,"dcValue2":"https://www.reddit.com/r/LaptopDeals/comments/lqglv5/guess_whats_coming_up_we_are_on_the_verge_of/","dcValue3":"t3_lqglv5","dcValue4":"ae4638099ccceaabee9bb6828307d5a1","dcValue5":1614081022},{"0":"And 8 more entries, set log level to debug to see all entries","dcValue1":5,"dcValue2":2,"dcValue3":2,"dcValue4":2,"dcValue5":1}]},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":217,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php","line":135,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/nextcloud/apps/news/lib/Db/NewsMapperV2.php","line":76,"function":"insert","class":"OCP\\AppFramework\\Db\\QBMapper","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/ItemServiceV2.php","line":102,"function":"insert","class":"OCA\\News\\Db\\NewsMapperV2","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/FeedServiceV2.php","line":302,"function":"insertOrUpdate","class":"OCA\\News\\Service\\ItemServiceV2","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/FeedServiceV2.php","line":341,"function":"fetch","class":"OCA\\News\\Service\\FeedServiceV2","type":"->","args":[{"items":[],"id":2,"__class__":"OCA\\News\\Db\\Feed"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/UpdaterService.php","line":55,"function":"fetchAll","class":"OCA\\News\\Service\\FeedServiceV2","type":"->","args":[]},{"file":"/var/www/nextcloud/apps/news/lib/Cron/UpdaterJob.php","line":71,"function":"update","class":"OCA\\News\\Service\\UpdaterService","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/BackgroundJob/Job.php","line":52,"function":"run","class":"OCA\\News\\Cron\\UpdaterJob","type":"->","args":[null]},{"file":"/var/www/nextcloud/lib/private/BackgroundJob/TimedJob.php","line":59,"function":"execute","class":"OC\\BackgroundJob\\Job","type":"->","args":[{"__class__":"OC\\BackgroundJob\\JobList"},{"__class__":"OC\\Log"}]},{"file":"/var/www/nextcloud/cron.php","line":127,"function":"execute","class":"OC\\BackgroundJob\\TimedJob","type":"->","args":[{"__class__":"OC\\BackgroundJob\\JobList"},{"__class__":"OC\\Log"}]}],"File":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractMySQLDriver.php","Line":106,"Previous":{"Exception":"Doctrine\\DBAL\\Driver\\PDOException","Message":"SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\\xF0\\x9F\\x8E\\x89 G...' for column `nextcloud`.`oc_news_items`.`title` at row 1","Code":"22007","Trace":[{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php","line":1062,"function":"execute","class":"Doctrine\\DBAL\\Driver\\PDOStatement","type":"->","args":[]},{"file":"/var/www/nextcloud/lib/private/DB/Connection.php","line":222,"function":"executeUpdate","class":"Doctrine\\DBAL\\Connection","type":"->","args":["INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",[true,"https://www.reddit.com/r/LaptopDeals/comments/lqglv5/guess_whats_coming_up_we_are_on_the_verge_of/","t3_lqglv5","ae4638099ccceaabee9bb6828307d5a1",1614081022,"And 8 more entries, set log level to debug to see all entries"],[5,2,2,2,1,"And 8 more entries, set log level to debug to see all entries"]]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php","line":209,"function":"executeUpdate","class":"OC\\DB\\Connection","type":"->","args":["INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(:dcValue1, :dcValue2, :dcValue3, :dcValue4, :dcValue5, :dcValue6, :dcValue7, :dcValue8, :dcValue9, :dcValue10, :dcValue11, :dcValue12, :dcValue13)",{"0":"And 8 more entries, set log level to debug to see all entries","dcValue1":true,"dcValue2":"https://www.reddit.com/r/LaptopDeals/comments/lqglv5/guess_whats_coming_up_we_are_on_the_verge_of/","dcValue3":"t3_lqglv5","dcValue4":"ae4638099ccceaabee9bb6828307d5a1","dcValue5":1614081022},{"0":"And 8 more entries, set log level to debug to see all entries","dcValue1":5,"dcValue2":2,"dcValue3":2,"dcValue4":2,"dcValue5":1}]},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":217,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php","line":135,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/nextcloud/apps/news/lib/Db/NewsMapperV2.php","line":76,"function":"insert","class":"OCP\\AppFramework\\Db\\QBMapper","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/ItemServiceV2.php","line":102,"function":"insert","class":"OCA\\News\\Db\\NewsMapperV2","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/FeedServiceV2.php","line":302,"function":"insertOrUpdate","class":"OCA\\News\\Service\\ItemServiceV2","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/FeedServiceV2.php","line":341,"function":"fetch","class":"OCA\\News\\Service\\FeedServiceV2","type":"->","args":[{"items":[],"id":2,"__class__":"OCA\\News\\Db\\Feed"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/UpdaterService.php","line":55,"function":"fetchAll","class":"OCA\\News\\Service\\FeedServiceV2","type":"->","args":[]},{"file":"/var/www/nextcloud/apps/news/lib/Cron/UpdaterJob.php","line":71,"function":"update","class":"OCA\\News\\Service\\UpdaterService","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/BackgroundJob/Job.php","line":52,"function":"run","class":"OCA\\News\\Cron\\UpdaterJob","type":"->","args":[null]},{"file":"/var/www/nextcloud/lib/private/BackgroundJob/TimedJob.php","line":59,"function":"execute","class":"OC\\BackgroundJob\\Job","type":"->","args":[{"__class__":"OC\\BackgroundJob\\JobList"},{"__class__":"OC\\Log"}]},{"file":"/var/www/nextcloud/cron.php","line":127,"function":"execute","class":"OC\\BackgroundJob\\TimedJob","type":"->","args":[{"__class__":"OC\\BackgroundJob\\JobList"},{"__class__":"OC\\Log"}]}],"File":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php","Line":129,"Previous":{"Exception":"PDOException","Message":"SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\\xF0\\x9F\\x8E\\x89 G...' for column `nextcloud`.`oc_news_items`.`title` at row 1","Code":"22007","Trace":[{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php","line":127,"function":"execute","class":"PDOStatement","type":"->","args":[null]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php","line":1062,"function":"execute","class":"Doctrine\\DBAL\\Driver\\PDOStatement","type":"->","args":[]},{"file":"/var/www/nextcloud/lib/private/DB/Connection.php","line":222,"function":"executeUpdate","class":"Doctrine\\DBAL\\Connection","type":"->","args":["INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",[true,"https://www.reddit.com/r/LaptopDeals/comments/lqglv5/guess_whats_coming_up_we_are_on_the_verge_of/","t3_lqglv5","ae4638099ccceaabee9bb6828307d5a1",1614081022,"And 8 more entries, set log level to debug to see all entries"],[5,2,2,2,1,"And 8 more entries, set log level to debug to see all entries"]]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php","line":209,"function":"executeUpdate","class":"OC\\DB\\Connection","type":"->","args":["INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(:dcValue1, :dcValue2, :dcValue3, :dcValue4, :dcValue5, :dcValue6, :dcValue7, :dcValue8, :dcValue9, :dcValue10, :dcValue11, :dcValue12, :dcValue13)",{"0":"And 8 more entries, set log level to debug to see all entries","dcValue1":true,"dcValue2":"https://www.reddit.com/r/LaptopDeals/comments/lqglv5/guess_whats_coming_up_we_are_on_the_verge_of/","dcValue3":"t3_lqglv5","dcValue4":"ae4638099ccceaabee9bb6828307d5a1","dcValue5":1614081022},{"0":"And 8 more entries, set log level to debug to see all entries","dcValue1":5,"dcValue2":2,"dcValue3":2,"dcValue4":2,"dcValue5":1}]},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":217,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php","line":135,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/nextcloud/apps/news/lib/Db/NewsMapperV2.php","line":76,"function":"insert","class":"OCP\\AppFramework\\Db\\QBMapper","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/ItemServiceV2.php","line":102,"function":"insert","class":"OCA\\News\\Db\\NewsMapperV2","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/FeedServiceV2.php","line":302,"function":"insertOrUpdate","class":"OCA\\News\\Service\\ItemServiceV2","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/FeedServiceV2.php","line":341,"function":"fetch","class":"OCA\\News\\Service\\FeedServiceV2","type":"->","args":[{"items":[],"id":2,"__class__":"OCA\\News\\Db\\Feed"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/UpdaterService.php","line":55,"function":"fetchAll","class":"OCA\\News\\Service\\FeedServiceV2","type":"->","args":[]},{"file":"/var/www/nextcloud/apps/news/lib/Cron/UpdaterJob.php","line":71,"function":"update","class":"OCA\\News\\Service\\UpdaterService","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/BackgroundJob/Job.php","line":52,"function":"run","class":"OCA\\News\\Cron\\UpdaterJob","type":"->","args":[null]},{"file":"/var/www/nextcloud/lib/private/BackgroundJob/TimedJob.php","line":59,"function":"execute","class":"OC\\BackgroundJob\\Job","type":"->","args":[{"__class__":"OC\\BackgroundJob\\JobList"},{"__class__":"OC\\Log"}]},{"file":"/var/www/nextcloud/cron.php","line":127,"function":"execute","class":"OC\\BackgroundJob\\TimedJob","type":"->","args":[{"__class__":"OC\\BackgroundJob\\JobList"},{"__class__":"OC\\Log"}]}],"File":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php","Line":127}},"CustomMessage":"Error while running background job (class: OCA\\News\\Cron\\UpdaterJob, arguments: )"},"userAgent":"--","version":"20.0.7.1","id":"60352c86ed578"}
anoymouserver commented 3 years ago

All three feeds work for me (NC 21, News 15.3.2, MariaDB 10.1.48). Did you enable the MySQL 4-byte support (utf8mb4) for your server?

Screenshot with the emojis ![grafik](https://user-images.githubusercontent.com/15173012/108888022-3e7c8d80-760b-11eb-9166-8d143eccd3ee.png)
Jdogzz commented 3 years ago

@anoymouserver Thanks for letting me know about that, I went through the process but it still threw errors in my log. However, I took one extra step and added a line to my database config file as noted here:

https://github.com/nextcloud/server/issues/20883#issuecomment-702833402

After that I added a test feed back in again and no more errors were generated. I will report back in the event the feeds stop updating again and errors return.

Jonny007-MKD commented 3 years ago

I also enabled utf8mb4 as described and it works now. Thanks!

LinAGKar commented 2 years ago

This can still happen even with the database set to utf8mb4, if a feed contains invalid UTF-8.

MCMic commented 2 years ago

I have the same issue, and indeed I do not have utf8mb4 yet, that's a limitation of yunohost package currently. Would it be possible to at least catch the error so that only the problematic feed do not update? Also having a list of feeds with errors somewhere would be useful.