update process failed with exit code 110

Thatoo commented 2 years ago

Describe the bug

Several feeds aren't updated beacuse it says : update process failed with exit code 110

Context

Hardware: Old laptop or computer
YunoHost version: 11
I have access to my server: Through SSH | through the webadmin | direct access via keyboard / screen | ...
Are you in a special context or did you perform some particular tweaking on your YunoHost instance?: no
If upgrading, current package version: 20220626~ynh1

Steps to reproduce

Several feeds are displayed red and are saying update process failed with exit code 110 and I can't read last entries even though I know some blogs have been updated. If I test on the demo website of ttrss, https://srv.tt-rss.org/tt-rss, they work.

Expected behavior

Should display new entries of blogs.

More

I have one feed with an other error : Failed to connect to xxx port 443 : connection refused even though I can connect to this feed by https in my webbrowser and it also works on https://srv.tt-rss.org/tt-rss And then, I still have the error with local feeds : https://github.com/YunoHost-Apps/ttrss_ynh/issues/134

ttrss was very useful to me on yunohost till recently but after loosing the possibility to fetch from local feeds and now all these errors, it became almost useless...

arofarn commented 2 years ago

Same here (Update process failed with exit code: 110) with some blogs and a lot of youtube channel (some are working but not the majority).

I tried without success :

to lower the refresh rate to once per day from once per hour in case of "flood ban"
reinstalling tt-rss with an import of the flux via OPML file (without tt-rss settings)

arofarn commented 2 years ago

In the feed debugger (LOG_EXTENDED, force refresh ON and force rehash ON), I have this king of message::

[07:36:02/913448] start
[07:36:02/913448] running HOOK_FETCH_FEED handlers...
[07:36:02/913448] feed data has not been modified by a plugin.
[07:36:02/913448] local cache will not be used for this feed
[07:36:02/913448] last unconditional update request: 
[07:36:02/913448] maximum allowed interval for conditional requests exceeded, forcing refetch
[07:36:02/913448] fetching https://www.youtube.com/feeds/videos.xml?channel_id=UCvDpfkniGG6WkX5L7DwyGaA (force_refetch: 1)...
[07:36:02/913448] [UrlHelper] fetching: https://www.youtube.com/feeds/videos.xml?channel_id=UCvDpfkniGG6WkX5L7DwyGaA
[07:36:02/913448] fetch done.
[07:36:02/913448] effective URL (after redirects): https://www.youtube.com/feeds/videos.xml?channel_id=UCvDpfkniGG6WkX5L7DwyGaA (IP: 172.217.18.206) 
[07:36:02/913448] server last modified: 
[07:36:02/913448] saving to local cache: cache/feeds/5908c6a723d24d06c96e83fd1acf4f0e0094b7f2.xml
[07:36:02/913448] running HOOK_FEED_FETCHED handlers...
[07:36:02/913448] feed data has not been modified by a plugin.
[07:36:02/913448] running HOOK_FEED_PARSED handlers...
[07:36:02/913448] language: simple
[07:36:02/913448] processing feed data...
[07:36:02/913448] site_url: https://www.youtube.com/channel/UCvDpfkniGG6WkX5L7DwyGaA
[07:36:02/913448] feed_title: Olivier Verdier
[07:36:02/913448] favicon: needs check: 1 is custom:  avg color: 
[07:36:02/913448] favicon: trying to update favicon...
[07:36:02/913448] [UrlHelper] fetching: https://www.youtube.com/channel/UCvDpfkniGG6WkX5L7DwyGaA
[07:36:02/913448] [UrlHelper] fetching: https://www.google.com/favicon.ico
[07:36:02/913448] favicon: https://www.google.com/favicon.ico looks valid, saving to feed-icons/46.ico
[07:36:02/913448] favicon: trying to calculate average color...
[07:36:02/913448] favicon: avg color: 
[07:36:02/913448] loading filters & labels...
Array
(
)
[07:36:02/913448] 0 filters loaded.
[07:36:02/913448] processing articles...
[07:36:02/913448] =================================================================================================================================
[07:36:02/913448] guid 2,yt:video:_UOM0Pz_Cgg (hash: {"ver":2,"uid":"2","hash":"SHA1:8538702c3e3ca19bdc132e689065614aaa4849be"} compat: SHA1:97fb5cb64a52095cc47c51c733c3441e60b905ce)
[07:36:02/913448] orig date: 1662320664 (2022-09-04 19:44:24)
[07:36:02/913448] title On fabrique 4 TRÉTEAUX aussi solides que pratiques 😉
[07:36:02/913448] link https://www.youtube.com/watch?v=_UOM0Pz_Cgg
[07:36:02/913448] language 
[07:36:02/913448] author Olivier Verdier
[07:36:02/913448] looking for tags...
[07:36:02/913448] tags found: 
[07:36:02/913448] done collecting data.
[07:36:02/913448] looking for enclosures...
[07:36:02/913448] article hash: 5cbaeebecfa1a6f7c6d3e57ddb01b1a4652694a1 [stored=]
[07:36:02/913448] hash differs, running HOOK_ARTICLE_FILTER handlers...
[07:36:02/913448] plugin data: 
[07:36:02/913448] matched filters: 
[07:36:02/913448] matched filter rules: 
[07:36:02/913448] filter actions: 
[07:36:02/913448] date: 1662320664 (2022/09/04 19:44:24)
[07:36:02/913448] num_comments: 0
[07:36:02/913448] article labels:
[07:36:02/913448] force catchup: 
[07:36:02/913448] base guid [2,yt:video:_UOM0Pz_Cgg or {"ver":2,"uid":"2","hash":"SHA1:8538702c3e3ca19bdc132e689065614aaa4849be"}] not found, creating...

It looks like an DB error when creating an none existing post entry ?

Axehaft commented 2 years ago

I have 4 feeds that are also showing the same error ... but seem to be updating normally. No clue what is causing this. If anybody needs more logs, let me know what you need and where to find it.

Veehem commented 2 years ago

I've many feeds with the same error

Axehaft commented 2 years ago

I suspect the YNH TTRSS version needs a new pull/update from the official ttrss site. It's beyond my skill/knowledge to do that though.

ericgaspar commented 2 years ago

@Axehaft ttrss_ynh as been updated today, we are using the latest pull in master branch...

Thatoo commented 2 years ago

It apparently didn't solve any problem I have been experimenting lately.

Axehaft commented 2 years ago

I updated as well ... still there for me too.

Axehaft commented 2 years ago

The Error 110 is shown here in the code: https://dev.tt-rss.org/fox/tt-rss/src/branch/master/update.php#L247

I still have no clue what it means though.

Axehaft commented 1 year ago

This issue continues to be unresolved. I asked the TTRSS maintainer for help:

"code 110 means there was an underlying exception during feed update, start looking in the event log. since you’re using an unsupported setup of some nonspecific version i won’t be able to help you any further."

So I can't get help from him. It seems to be something on the YNH version of TTRSS as there is no mention of any issue on the main TTRSS forum. Interesting note: I get the same issue if I use RSS Bridge to create a feed of the sites with errors as well. So maybe something when fetching new content?

Each one that fails seems to stop here:

[13:51:38/450421] force catchup: [13:51:38/450421] base guid [2,https://www.instagram.com/p/Cj8TxoLgFrB/ or {"ver":2,"uid":"2","hash":"SHA1:5521392c4585a5e43d61a3ec825fc66b216aa707"}] not found, creating...

[13:50:45/558284] force catchup: [13:50:45/558284] base guid [2,https://www.youtube.com/watch?v=dafoHfQFnwQ or {"ver":2,"uid":"2","hash":"SHA1:4ae1269ef9fc7814cf6c9e1bf1ac0b95bb144f6b"}] not found, creating...

[13:53:57/450424] force catchup: [13:53:57/450424] base guid [2,https://t.me/ChefPeteEvans/12697 or {"ver":2,"uid":"2","hash":"SHA1:e5f18f3f88bd32e0e0b16f110a13e51e2866960a"}] not found, creating...

Each log stop at this point. Does that help anyone in figuring this out?

Veehem commented 1 year ago

I'm also desperately waiting for a solution... and having the same problem with feeds coming from RSS Bridge

Polonco commented 1 year ago

I'm having the same problem with some of the feeds. Anybody found a fix or a workaround?

Following the error message displayed in /ttrss/prefs.php?tab=system when I force refetch and rehash for that feed.

`

Error | Filename | Message | User | Date -- | -- | -- | -- | -- E_USER_WARNING (512) | /var/www/ttrss/classes/rssutils.php:1027 | SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\xF0\x9F\x98\x88" ...' for column `ttrss`.`ttrss_entries`.`content` at row 1 #0 /var/www/ttrss/classes/rssutils.php(1027): PDOStatement->execute() #1 /var/www/ttrss/update.php(238): RSSUtils::update_rss_feed() #2 {main} | | 18:06 E_USER_WARNING (512) | /var/www/ttrss/classes/rssutils.php:1027 | SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\xF0\x9F\x8E\xA5 a...' for column `ttrss`.`ttrss_entries`.`content` at row 1 #0 /var/www/ttrss/classes/rssutils.php(1027): PDOStatement->execute() #1 /var/www/ttrss/update.php(238): RSSUtils::update_rss_feed() #2 {main} | | 18:06 E_USER_WARNING (512) | /var/www/ttrss/classes/rssutils.php:1027 | SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\xF0\x9F\x98\x88 1...' for column `ttrss`.`ttrss_entries`.`content` at row 1 #0 /var/www/ttrss/classes/rssutils.php(1027): PDOStatement->execute() #1 /var/www/ttrss/update.php(238): RSSUtils::update_rss_feed() #2 {main} | | 18:06 E_ERROR (1) | classes/rssutils.php:1027 | Uncaught PDOException: SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\xF0\x9F\x91\x80execute() #1 /var/www/ttrss/classes/feeds.php(753): RSSUtils::update_rss_feed() #2 /var/www/ttrss/backend.php(136): Feeds->updatedebugger() #3 {main} thrown Remote IP: REMOVED.PUBLIC.IP.ADDRESS Request URI: /ttrss/backend.php User agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/109.0 | admin | 18:06 E_USER_WARNING (512) | /var/www/ttrss/classes/rssutils.php:1027 | SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\xF0\x9F\x92\xA6" ...' for column `ttrss`.`ttrss_entries`.`content` at row 1 #0 /var/www/ttrss/classes/rssutils.php(1027): PDOStatement->execute() #1 /var/www/ttrss/update.php(238): RSSUtils::update_rss_feed() #2 {main} | | 18:06 E_USER_WARNING (512) | /var/www/ttrss/classes/rssutils.php:1027 | SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\xF0\x9F\x8E\x89\xE2\x9A...' for column `ttrss`.`ttrss_entries`.`content` at row 1 #0 /var/www/ttrss/classes/rssutils.php(1027): PDOStatement->execute() #1 /var/www/ttrss/update.php(238): RSSUtils::update_rss_feed() #2 {main} | | 18:06 E_USER_WARNING (512) | /var/www/ttrss/classes/rssutils.php:1027 | SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\xF0\x9F\x8C\x88" ...' for column `ttrss`.`ttrss_entries`.`content` at row 1 #0 /var/www/ttrss/classes/rssutils.php(1027): PDOStatement->execute() #1 /var/www/ttrss/update.php(238): RSSUtils::update_rss_feed() #2 {main} | | 18:06 E_USER_WARNING (512) | /var/www/ttrss/classes/rssutils.php:1027 | SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\xF0\x9F\x91\x80execute() #1 /var/www/ttrss/update.php(238): RSSUtils::update_rss_feed() #2 {main}

`

El-Gavy commented 1 year ago

Same issue here for a lot of feeds (app version : 20230107~ynh1)

nathanael-h commented 1 year ago

Looking at this issue, don't you think it might be caused by emojis in title, or text? \xF0\x9F\x91\x80 is :eyes: eyes emoji

Looking at sources, I see that the character set used it UTF8 https://dev.tt-rss.org/tt-rss/tt-rss/src/branch/master/sql/mysql/schema.sql#L2

I know that Nextcloud introduced a change to fix a related issue : https://docs.nextcloud.com/server/latest/admin_manual/configuration_database/mysql_4byte_support.html

I'll try to use the official docker-compose way of deploying ttrss and see if I can reproduce it. Maybe it is an upstream issue.

Thatoo commented 1 year ago

I don't know but the problem appear suddenly after upgrading to this version 20220626~ynh1, I never had this issue in the previous 5 years and now I have it for plenty of rss feeds, so many that ttrss became almost useless to me.

nathanael-h commented 1 year ago

Yeah, the same for me. I was kinda thinking "wow I catch up all the non read posts I had for months" and then realized that I didn't read any post from certain feeds for weeks 😮

So I started the docker-compose stack locally, and there is no problem retrieving feed with emoji, whereas in the ttrss_ynh app.

Here is an eg with Matrix.org blog feed :

So, i found few pages, threads and commits about this :

Another thing I noticed, is that the recommended db is PostgreSQL, whereas we use MariaDB. Maybe, switching to PgSQL might help. This non yet merged, but still updated, PR in Nextcloud_ynh repo might guide us : https://github.com/YunoHost-Apps/nextcloud_ynh/pull/125

Maybe a change of character set might fix the issue, but I don't know enough things about db and encoding.

What do you think? Ping @ericgaspar

ericgaspar commented 1 year ago

There is already a PostgreSQL branch https://github.com/YunoHost-Apps/ttrss_ynh/tree/PostgreSQL. I am not concidering doing a migration script for migrating from Mariadb to PostgreSQL as the option to export OPML, remove TTRSS and reinstall TTRSS with PostgreSQL + reimport OPML file seems to be the easiest option. I'm curious to hear what you think of migrating to PostgreSQL like this.

nathanael-h commented 1 year ago

I saw this PR changing the carset in config.php file https://github.com/YunoHost-Apps/ttrss_ynh/pull/132 Shouldn't we have a script to change the carset in exsiting installation while upgrading?

Oh wow, super this #150 PR :) I think this is a good path, I'd suggest we also use the data migration plugin, to migrate user data (cf. https://tt-rss.org/wiki/FAQ)

export OPML (is it possible to export one global OPML, or is it per user? I did not find)
export per user data using https://dev.tt-rss.org/tt-rss/ttrss-data-migration/wiki
delete TTRSS
install TTRSS using PgSQL
import OPML
import per user data

Veehem commented 1 year ago

Anyway if there is a solution (I'm also waiting), please give a clear way to manage it. Everybody is not code fluent

arofarn commented 1 year ago

Is it possible to install both "old" ttrss with MariaDb and the new ttrss with postgreSQL on the same yunohost instance for testing without removing what works partially well ? Or will it collide and crash the whole thing ?

ericgaspar commented 1 year ago

It should be possible as multi_instance = true

arofarn commented 1 year ago

It didn't go as expected : the old TTRSS became unreachable (redirect to yunohost main page) and I had to uninstall both before restoring my old TTRSS (with MariaDb) from a backup... As I didn't export the OPML before the installation of TTRSS with PostgreSQL, I stop here for today.

loulecrivain commented 1 year ago

Hello, I've done some debugging and apparently this is a collation problem. The app is configured to use UTF8MB4 collation, which is also the case for the database, right...

But ! All the tables are configured as utf8_general_ci (excerpt for ttrss_entries)

+---------------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+------------+-----------------+----------+----------------+---------+------------------+-----------+
| Name          | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time         | Update_time         | Check_time | Collation       | Checksum | Create_options | Comment | Max_index_length | Temporary |
+---------------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+------------+-----------------+----------+----------------+---------+------------------+-----------+
| ttrss_entries | InnoDB |      10 | Dynamic    | 1225 |           6272 |     7684096 |               0 |       327680 |  25165824 |           4794 | 2023-06-24 00:46:58 | 2023-06-30 12:08:11 | NULL       | utf8_general_ci |     NULL |                |         |                0 | N         |
+---------------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+------------+-----------------+----------+----------------+---------+------------------+-----------+
1 row in set (0,000 sec)

which are responsible for the PDOExceptions.

@ericgaspar do you want me to do a PR ? not sure how we should fix this.

loulecrivain commented 1 year ago

hello, source of the bug is now confirmed. I ran the following command in mariadb:

ALTER TABLE ttrss_entries CONVERT TO CHARACTER SET 'utf8mb4' COLLATE 'utf8mb4_general_ci';

and this fixed all my feeds. feel free to @ me if you need more debugging info. thank you.

ericgaspar commented 1 year ago

Is adding this to install and upgrade script do the job?

ynh_mysql_connect_as --user=$db_user --password="$db_pwd" --database=$db_name \
    <<< "ALTER DATABASE $db_name CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;"

loulecrivain commented 1 year ago

@ericgaspar that'd be quite the hack, but it could be an ok first fix given how longstanding this issue has been.

I'll fork the repo to do an install in a fresh vm and let you know

loulecrivain commented 1 year ago

@ericgaspar I tested with your latest release, since you made the changes before me. charset and collation change is not cascading, the tables are still in the utf8_general_ci collation and utf8 charset.

so the bug is still there. I'd like to do a PR for you but I don't have the time atm. is it okay if I come back to it in say, 2 weeks or so ?

ballinger commented 1 year ago

that also solved it for me. was not able to log into the db in the terminal, did it with myphpadmin. did not have the option for "utf8mb4" that was just the group name for all the utf8mb4's. i chose "utf8mb4_unicode_ci" which also works.

arofarn commented 9 months ago

I did the same as ballinger with phpMyAdmin : I choose "utf8mb4_unicode_ci" instead of "utf8mb4_general_ci" and checked to apply to all the tables and all the columns in the tables.

That solves all the "code 110" fails.

ericgaspar commented 9 months ago

Closing following the transition to PostgreSQL

YunoHost-Apps / ttrss_ynh