cheywood commented 13 hours ago

IMPORTANT

Read and tick the following checkbox after you have created the issue or place an x inside the brackets ;)

[x] I have read the CONTRIBUTING.md and followed the provided tips
[x] I accept that the issue will be closed without comment if I do not check here
[x] I accept that the issue will be closed without comment if I do not fill out all items in the issue template.

Explain the Problem

Adding Mastodon feeds appears to fail during import; only a number of older posts appear to be retained before the process stops. I've seen it happen on a number of different Mastodon instances and accounts. (Other RSS readers are able to process these feeds)

Steps to Reproduce

Click Subscribe
Enter eg. https://mastodon.social/@Mastodon
Witness a few older entries get imported
See 500 error
There's reference to a malformed xml in the server logs plus, perhaps more critically An exception oc curred while executing a query: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'media_description' in 'field list'

System Information

News app version: 25.1.0
Nextcloud version: 29.0.9
Browser and version: Firefox 133.0
Cron type: Cron
PHP version: 8.3.14
Database and version: MariaDB 10.5.23
OS and version: Client or server? Either way Debian, 11 on server, testing on client.

Contents of nextcloud/data/nextcloud.log

```json {"reqId":"a3VHC6C1q1HV7cpV7qJE","level":2,"time":"2024-12-04T08:47:09+00:00","remoteAddr":"????","user":"username","app":"news","method":"POST","url":"/nextcloud/apps/news/feeds","message":"https://mastodon.social/@Mastodon read error : malformed xml string. parsing error : DOMDocument::loadXML(): Opening and ending tag mismatch: meta line 50 and head in Entity, line: 54 (2)","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:133.0) Gecko/20100101 Firefox/133.0","version":"29.0.9.2","data":{"app":"news"}} {"reqId":"a3VHC6C1q1HV7cpV7qJE","level":2,"time":"2024-12-04T08:47:09+00:00","remoteAddr":"????","user":"username","app":"news","method":"POST","url":"/nextcloud/apps/news/feeds","message":"No valid feed found at URL, attempting auto discovery","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:133.0) Gecko/20100101 Firefox/133.0","version":"29.0.9.2","data":{"app":"news"}} {"reqId":"a3VHC6C1q1HV7cpV7qJE","level":3,"time":"2024-12-04T08:47:11+00:00","remoteAddr":"????","user":"username","app":"index","method":"POST","url":"/nextcloud/apps/news/feeds","message":"An exception occurred while executing a query: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'media_description' in 'field list'","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:133.0) Gecko/20100101 Firefox/133.0","version":"29.0.9.2","exception":{"Exception":"OC\\DB\\Exceptions\\DbalException","Message":"An exception occurred while executing a query: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'media_description' in 'field list'","Code":1054,"Trace":[{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":328,"function":"wrap","class":"OC\\DB\\Exceptions\\DbalException","type":"::"},{"file":"/var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php","line":137,"function":"executeStatement","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud/apps/news/lib/Db/NewsMapperV2.php","line":76,"function":"insert","class":"OCP\\AppFramework\\Db\\QBMapper","type":"->"},{"file":"/var/www/nextcloud/apps/news/lib/Service/ItemServiceV2.php","line":104,"function":"insert","class":"OCA\\News\\Db\\NewsMapperV2","type":"->"},{"file":"/var/www/nextcloud/apps/news/lib/Service/FeedServiceV2.php","line":334,"function":"insertOrUpdate","class":"OCA\\News\\Service\\ItemServiceV2","type":"->"},{"file":"/var/www/nextcloud/apps/news/lib/Controller/FeedController.php","line":157,"function":"fetch","class":"OCA\\News\\Service\\FeedServiceV2","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":232,"function":"create","class":"OCA\\News\\Controller\\FeedController","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":138,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/App.php","line":184,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/Route/Router.php","line":331,"function":"main","class":"OC\\AppFramework\\App","type":"::"},{"file":"/var/www/nextcloud/lib/base.php","line":1058,"function":"match","class":"OC\\Route\\Router","type":"->"},{"file":"/var/www/nextcloud/index.php","line":49,"function":"handleRequest","class":"OC","type":"::"}],"File":"/var/www/nextcloud/lib/private/DB/Exceptions/DbalException.php","Line":71,"Previous":{"Exception":"Doctrine\\DBAL\\Exception\\InvalidFieldNameException","Message":"An exception occurred while executing a query: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'media_description' in 'field list'","Code":1054,"Trace":[{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php","line":1943,"function":"convert","class":"Doctrine\\DBAL\\Driver\\API\\MySQL\\ExceptionConverter","type":"->"},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php","line":1885,"function":"handleDriverException","class":"Doctrine\\DBAL\\Connection","type":"->"},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php","line":1213,"function":"convertExceptionDuringQuery","class":"Doctrine\\DBAL\\Connection","type":"->"},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Connections/PrimaryReadReplicaConnection.php","line":292,"function":"executeStatement","class":"Doctrine\\DBAL\\Connection","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/Connection.php","line":383,"function":"executeStatement","class":"Doctrine\\DBAL\\Connections\\PrimaryReadReplicaConnection","type":"->"},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php","line":393,"function":"executeStatement","class":"OC\\DB\\Connection","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":280,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":326,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php","line":137,"function":"executeStatement","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud/apps/news/lib/Db/NewsMapperV2.php","line":76,"function":"insert","class":"OCP\\AppFramework\\Db\\QBMapper","type":"->"},{"file":"/var/www/nextcloud/apps/news/lib/Service/ItemServiceV2.php","line":104,"function":"insert","class":"OCA\\News\\Db\\NewsMapperV2","type":"->"},{"file":"/var/www/nextcloud/apps/news/lib/Service/FeedServiceV2.php","line":334,"function":"insertOrUpdate","class":"OCA\\News\\Service\\ItemServiceV2","type":"->"},{"file":"/var/www/nextcloud/apps/news/lib/Controller/FeedController.php","line":157,"function":"fetch","class":"OCA\\News\\Service\\FeedServiceV2","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":232,"function":"create","class":"OCA\\News\\Controller\\FeedController","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":138,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/App.php","line":184,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/Route/Router.php","line":331,"function":"main","class":"OC\\AppFramework\\App","type":"::"},{"file":"/var/www/nextcloud/lib/base.php","line":1058,"function":"match","class":"OC\\Route\\Router","type":"->"},{"file":"/var/www/nextcloud/index.php","line":49,"function":"handleRequest","class":"OC","type":"::"}],"File":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/API/MySQL/ExceptionConverter.php","Line":67,"Previous":{"Exception":"Doctrine\\DBAL\\Driver\\PDO\\Exception","Message":"SQLSTATE[42S22]: Column not found: 1054 Unknown column 'media_description' in 'field list'","Code":1054,"Trace":[{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/PDO/Statement.php","line":132,"function":"new","class":"Doctrine\\DBAL\\Driver\\PDO\\Exception","type":"::"},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php","line":1207,"function":"execute","class":"Doctrine\\DBAL\\Driver\\PDO\\Statement","type":"->"},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Connections/PrimaryReadReplicaConnection.php","line":292,"function":"executeStatement","class":"Doctrine\\DBAL\\Connection","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/Connection.php","line":383,"function":"executeStatement","class":"Doctrine\\DBAL\\Connections\\PrimaryReadReplicaConnection","type":"->"},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php","line":393,"function":"executeStatement","class":"OC\\DB\\Connection","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":280,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":326,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php","line":137,"function":"executeStatement","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud/apps/news/lib/Db/NewsMapperV2.php","line":76,"function":"insert","class":"OCP\\AppFramework\\Db\\QBMapper","type":"->"},{"file":"/var/www/nextcloud/apps/news/lib/Service/ItemServiceV2.php","line":104,"function":"insert","class":"OCA\\News\\Db\\NewsMapperV2","type":"->"},{"file":"/var/www/nextcloud/apps/news/lib/Service/FeedServiceV2.php","line":334,"function":"insertOrUpdate","class":"OCA\\News\\Service\\ItemServiceV2","type":"->"},{"file":"/var/www/nextcloud/apps/news/lib/Controller/FeedController.php","line":157,"function":"fetch","class":"OCA\\News\\Service\\FeedServiceV2","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":232,"function":"create","class":"OCA\\News\\Controller\\FeedController","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":138,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/App.php","line":184,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/Route/Router.php","line":331,"function":"main","class":"OC\\AppFramework\\App","type":"::"},{"file":"/var/www/nextcloud/lib/base.php","line":1058,"function":"match","class":"OC\\Route\\Router","type":"->"},{"file":"/var/www/nextcloud/index.php","line":49,"function":"handleRequest","class":"OC","type":"::"}],"File":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/PDO/Exception.php","Line":28,"Previous":{"Exception":"PDOException","Message":"SQLSTATE[42S22]: Column not found: 1054 Unknown column 'media_description' in 'field list'","Code":"42S22","Trace":[{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/PDO/Statement.php","line":130,"function":"execute","class":"PDOStatement","type":"->"},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php","line":1207,"function":"execute","class":"Doctrine\\DBAL\\Driver\\PDO\\Statement","type":"->"},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Connections/PrimaryReadReplicaConnection.php","line":292,"function":"executeStatement","class":"Doctrine\\DBAL\\Connection","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/Connection.php","line":383,"function":"executeStatement","class":"Doctrine\\DBAL\\Connections\\PrimaryReadReplicaConnection","type":"->"},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php","line":393,"function":"executeStatement","class":"OC\\DB\\Connection","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":280,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":326,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php","line":137,"function":"executeStatement","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud/apps/news/lib/Db/NewsMapperV2.php","line":76,"function":"insert","class":"OCP\\AppFramework\\Db\\QBMapper","type":"->"},{"file":"/var/www/nextcloud/apps/news/lib/Service/ItemServiceV2.php","line":104,"function":"insert","class":"OCA\\News\\Db\\NewsMapperV2","type":"->"},{"file":"/var/www/nextcloud/apps/news/lib/Service/FeedServiceV2.php","line":334,"function":"insertOrUpdate","class":"OCA\\News\\Service\\ItemServiceV2","type":"->"},{"file":"/var/www/nextcloud/apps/news/lib/Controller/FeedController.php","line":157,"function":"fetch","class":"OCA\\News\\Service\\FeedServiceV2","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":232,"function":"create","class":"OCA\\News\\Controller\\FeedController","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":138,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/App.php","line":184,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/Route/Router.php","line":331,"function":"main","class":"OC\\AppFramework\\App","type":"::"},{"file":"/var/www/nextcloud/lib/base.php","line":1058,"function":"match","class":"OC\\Route\\Router","type":"->"},{"file":"/var/www/nextcloud/index.php","line":49,"function":"handleRequest","class":"OC","type":"::"}],"File":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/PDO/Statement.php","Line":130}}},"message":"An exception occurred while executing a query: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'media_description' in 'field list'","exception":{},"CustomMessage":"An exception occurred while executing a query: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'media_description' in 'field list'"}} ```

Contents of Browser Error Console

Read http://ggnome.com/wiki/Using_The_Browser_Error_Console if you are unsure what to put here ``` XHRPOST https://SERVER/nextcloud/apps/news/feeds [HTTP/2 500 2957ms] Object { stack: Getter & Setter, message: Getter & Setter, name: Getter & Setter, code: Getter & Setter, config: Getter & Setter, request: Getter & Setter, response: Getter & Setter, status: Getter & Setter, … } feed.ts:97:11 ```

SMillerDev commented 11 hours ago

Please include those server logs as the template requested

cheywood commented 9 hours ago

> describe oc_news_items;
+-----------------+---------------------+------+-----+---------+----------------+
| Field           | Type                | Null | Key | Default | Extra          |
+-----------------+---------------------+------+-----+---------+----------------+
| id              | bigint(20) unsigned | NO   | PRI | NULL    | auto_increment |
| guid_hash       | varchar(32)         | NO   | MUL | NULL    |                |
| fingerprint     | varchar(32)         | YES  | MUL | NULL    |                |
| content_hash    | varchar(32)         | YES  |     | NULL    |                |
| rtl             | tinyint(1)          | NO   |     | 0       |                |
| search_index    | longtext            | YES  |     | NULL    |                |
| guid            | longtext            | NO   |     | NULL    |                |
| url             | longtext            | YES  |     | NULL    |                |
| title           | longtext            | YES  |     | NULL    |                |
| author          | longtext            | YES  |     | NULL    |                |
| pub_date        | bigint(20)          | YES  |     | NULL    |                |
| body            | longtext            | YES  |     | NULL    |                |
| enclosure_mime  | longtext            | YES  |     | NULL    |                |
| enclosure_link  | longtext            | YES  |     | NULL    |                |
| feed_id         | bigint(20) unsigned | NO   | MUL | NULL    |                |
| unread          | tinyint(1)          | NO   |     | 0       |                |
| starred         | tinyint(1)          | NO   |     | 0       |                |
| last_modified   | bigint(20)          | YES  | MUL | 0       |                |
| categories_json | longtext            | YES  |     | NULL    |                |
| shared_by       | varchar(64)         | YES  |     | NULL    |                |
+-----------------+---------------------+------+-----+---------+----------------+
20 rows in set (0.003 sec)

nextcloud / news

Mastodon feeds are mishandled #2966

IMPORTANT

Explain the Problem

Steps to Reproduce

System Information