search

joomla / joomla-cms

Home of the Joomla! Content Management System
https://www.joomla.org
GNU General Public License v2.0
4.77k stars 3.65k forks source link

Update Sites - Joomla! Core (Could not load root metadata) #43287

Closed alecrespi78 closed 7 months ago

alecrespi78 commented 7 months ago

Steps to reproduce the issue

Upgrade from 5.0.3 to 5.1.0

Update Sites - Joomla! Core (enabled) https://update.joomla.org/cms/

Expected result

Working checking notifications in Home Dashboard

Actual result

Notifications error (red icons) on Extensions / Overrides / Pull requests

Unknown extensions … Error on checking overrides. Unknown requests …

An error has occurred. 0 Could not load root metadata.

System information (as much as possible)

PHP Built On Linux webxc201s02.ad.aruba.it 3.10.0-1160.92.1.el7.x86_64 #1 SMP Tue Jun 20 11:48:01 UTC 2023 x86_64 Database Type mysql Database Version 8.0.32-24 PHP Version 8.2.13 Web Server Apache Joomla! Version Joomla! 5.1.0 Stable [ Kudumisha ] 16-April-2024 16:00 GMT Joomla Backward Compatibility Plugin Enabled ()

Additional comments

This is a part of the log...

2024-04-17T09:55:04+00:00 CRITICAL xxx.xxx.xxx.xxx error Uncaught Throwable of type LogicException thrown with message "Could not load root metadata.". Stack trace: #0 [ROOT]/libraries/vendor/php-tuf/php-tuf/src/Client/Updater.php(138): Tuf\Metadata\StorageBase->getRoot()

1 [ROOT]/libraries/src/TUF/TufFetcher.php(117): Tuf\Client\Updater->refresh()

2 [ROOT]/libraries/src/Updater/Adapter/TufAdapter.php(84): Joomla\CMS\TUF\TufFetcher->getValidUpdate()

3 [ROOT]/libraries/src/Updater/Adapter/TufAdapter.php(50): Joomla\CMS\Updater\Adapter\TufAdapter->getUpdateTargets()

4 [ROOT]/libraries/src/Updater/Updater.php(263): Joomla\CMS\Updater\Adapter\TufAdapter->findUpdate()

5 [ROOT]/libraries/src/Updater/Updater.php(164): Joomla\CMS\Updater\Updater->getUpdateObjectsForSite()

6 [ROOT]/administrator/components/com_installer/src/Model/UpdateModel.php(275): Joomla\CMS\Updater\Updater->findUpdates()

7 [ROOT]/administrator/components/com_installer/src/Controller/UpdateController.php(161): Joomla\Component\Installer\Administrator\Model\UpdateModel->findUpdates()

8 [ROOT]/libraries/src/MVC/Controller/BaseController.php(730): Joomla\Component\Installer\Administrator\Controller\UpdateController->ajax()

9 [ROOT]/libraries/src/Dispatcher/ComponentDispatcher.php(143): Joomla\CMS\MVC\Controller\BaseController->execute()

10 [ROOT]/libraries/src/Component/ComponentHelper.php(361): Joomla\CMS\Dispatcher\ComponentDispatcher->dispatch()

11 [ROOT]/libraries/src/Application/AdministratorApplication.php(150): Joomla\CMS\Component\ComponentHelper::renderComponent()

12 [ROOT]/libraries/src/Application/AdministratorApplication.php(195): Joomla\CMS\Application\AdministratorApplication->dispatch()

13 [ROOT]/libraries/src/Application/CMSApplication.php(306): Joomla\CMS\Application\AdministratorApplication->doExecute()

14 [ROOT]/administrator/includes/app.php(58): Joomla\CMS\Application\CMSApplication->execute()

15 [ROOT]/administrator/index.php(32): require_once('...')

16 {main}

alecrespi78 commented 7 months ago

Disabling the Update Sites >>> Joomla! Core plugin bring back to green all 3 buttons (home dashboard). image image

This allow searching and updating installed extensions (with this warning message "Some update sites are disabled. You may want to check the Update Sites Manager") but if you check for Joomla update it automatically enable the "Joomla! Core plugin" and shows this warning: (success) Checked for updates. An error has occurred. 0 Could not load root metadata. image image

Returning to Dashboars then shows the 3 buttons with red background. image image

SniperSister commented 7 months ago

Upgrade from 5.0.3 to 5.1.0

any problems during the update? How was it applied?

Could you please provide the content of the #__tuf_metadata table?

brianteeman commented 7 months ago

Please paste the contents of the update log

alecrespi78 commented 7 months ago

Update completed successfully with the internal wizard / guided procedure.

I've done that in several other sites without issues (same extensions installed).

__tuf_metadata table was empty.

I cloned it from a similar installation and everything worked! It was easy after all, thank you very much!

brianteeman commented 7 months ago

This means that the update was NOT successful. Please pust the update log

alecrespi78 commented 7 months ago

This is the content of the "/administrator/logs/joomla_update.php" file.

Date: 2024-04-17 06:32:52 UTC

Software: Joomla! 5.0.3 Stable [ Kuboresha ] 20-February-2024 16:00 GMT

Fields: datetime priority clientip category message

2024-04-17T06:32:52+00:00 INFO 195.210.40.226 update Test logging 2024-04-17T06:32:52+00:00 INFO 195.210.40.226 update Update started by user alecrespi (223). Old version is 5.0.3. 2024-04-17T06:32:53+00:00 INFO 195.210.40.226 update Downloading update file from https://s3-us-west-2.amazonaws.com/joomla-official-downloads/joomladownloads/joomla5/Joomla_5.1.0-Stable-Update_Package.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA6LXDJLNUINX2AVMH%2F20240417%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20240417T063252Z&X-Amz-Expires=60&X-Amz-SignedHeaders=host&X-Amz-Signature=4103a267307c44ecffb07f9d49269937adf505822c043317dfeeb2faf9f745bd. 2024-04-17T06:32:57+00:00 INFO 195.210.40.226 update File Joomla_5.1.0-Stable-Update_Package.zip downloaded. 2024-04-17T06:32:58+00:00 INFO 195.210.40.226 update Starting installation of new version. 2024-04-17T09:32:53+00:00 INFO 195.210.40.226 update Test logging 2024-04-17T09:32:53+00:00 INFO 195.210.40.226 update Uploading update file 2024-04-17T09:33:57+00:00 INFO 195.210.40.226 update File [ROOT][TMP]/juI32R3Y downloaded. 2024-04-17T09:33:58+00:00 INFO 195.210.40.226 update Starting installation of new version. 2024-04-17T09:34:54+00:00 INFO 195.210.40.226 update Finalising installation. 2024-04-17T09:34:54+00:00 INFO 195.210.40.226 update Start of SQL updates. 2024-04-17T09:34:54+00:00 INFO 195.210.40.226 update The current database version (schema) is 5.1.0-2024-03-28. 2024-04-17T09:34:54+00:00 INFO 195.210.40.226 update End of SQL updates. 2024-04-17T09:34:54+00:00 INFO 195.210.40.226 update Uninstalling extensions 2024-04-17T09:34:54+00:00 INFO 195.210.40.226 update Deleting removed files and folders. 2024-04-17T09:34:57+00:00 INFO 195.210.40.226 update Cleaning up after installation. 2024-04-17T09:34:57+00:00 INFO 195.210.40.226 update Update to version 5.1.0 is complete.

brianteeman commented 7 months ago

That explains why the TUF database did not contain anything and it also shows that you do not have a completed update,

No SQL updates took place!!

This is what I would have expected to see. @richard67 any thoughts?

Date: 2024-04-17 10:09:35 UTC

Software: Joomla! 5.0.3 Stable [ Kuboresha ] 20-February-2024 16:00 GMT

datetime priority clientip category message
2024-04-17T10:09:35+00:00 INFO 193.203.14.65 update Test logging
2024-04-17T10:09:35+00:00 INFO 193.203.14.65 update Update started by user Brian Teeman (491). Old version is 5.0.3.
2024-04-17T10:09:36+00:00 INFO 193.203.14.65 update Downloading update file from https://s3-us-west-2.amazonaws.com/joomla-official-downloads/joomladownloads/joomla5/Joomla_5.1.0-Stable-Update_Package.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA6LXDJLNUINX2AVMH%2F20240417%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20240417T100936Z&X-Amz-Expires=60&X-Amz-SignedHeaders=host&X-Amz-Signature=65572a314f5a02af89c33450a7e2cfdb2b894a4397f07f1a6ee82a57f2836c9a.
2024-04-17T10:09:43+00:00 INFO 193.203.14.65 update File Joomla_5.1.0-Stable-Update_Package.zip downloaded.
2024-04-17T10:10:02+00:00 INFO 193.203.14.65 update Starting installation of new version.
2024-04-17T10:10:08+00:00 INFO 193.203.14.65 update Finalising installation.
2024-04-17T10:10:08+00:00 INFO 193.203.14.65 update Start of SQL updates.
2024-04-17T10:10:08+00:00 INFO 193.203.14.65 update The current database version (schema) is 5.0.0-2023-09-11.
2024-04-17T10:10:08+00:00 INFO 193.203.14.65 update Ran query from file 5.1.0-2023-11-28. Query text: INSERT INTO #__extensions (package_id, name, type, element, folder, .
2024-04-17T10:10:08+00:00 INFO 193.203.14.65 update Ran query from file 5.1.0-2024-01-04. Query text: ALTER TABLE #__fields_values MODIFY value MEDIUMTEXT;.
2024-04-17T10:10:08+00:00 INFO 193.203.14.65 update Ran query from file 5.1.0-2024-02-10. Query text: ALTER TABLE #__guidedtour_steps ADD COLUMN params text NULL ;.
2024-04-17T10:10:08+00:00 INFO 193.203.14.65 update Ran query from file 5.1.0-2024-02-24. Query text: CREATE TABLE IF NOT EXISTS #__tuf_metadata ( id int NOT NULL AUTO_INCREMEN.
2024-04-17T10:10:08+00:00 INFO 193.203.14.65 update Ran query from file 5.1.0-2024-02-24. Query text: INSERT INTO #__tuf_metadata (update_site_id, root) SELECT ue.`updatesite.
2024-04-17T10:10:08+00:00 INFO 193.203.14.65 update Ran query from file 5.1.0-2024-02-24. Query text: UPDATE #__update_sites SET type = 'tuf', location = 'https://update.joo.
2024-04-17T10:10:08+00:00 INFO 193.203.14.65 update Ran query from file 5.1.0-2024-02-25. Query text: INSERT INTO #__extensions (package_id, name, type, element, folder, .
2024-04-17T10:10:08+00:00 INFO 193.203.14.65 update Ran query from file 5.1.0-2024-03-08. Query text: INSERT IGNORE INTO #__guidedtour_steps (tour_id, title, published, `desc.
2024-04-17T10:10:08+00:00 INFO 193.203.14.65 update Ran query from file 5.1.0-2024-03-08. Query text: ALTER TABLE #__guidedtours ADD COLUMN autostart int NOT NULL DEFAULT 0 ;.
2024-04-17T10:10:08+00:00 INFO 193.203.14.65 update Ran query from file 5.1.0-2024-03-08. Query text: INSERT IGNORE INTO #__guidedtours (title, uid, description, ordering, .
2024-04-17T10:10:08+00:00 INFO 193.203.14.65 update Ran query from file 5.1.0-2024-03-08. Query text: UPDATE #__guidedtour_steps SET tour_id = LAST_INSERT_ID() WHERE tour_id=0;.
2024-04-17T10:10:08+00:00 INFO 193.203.14.65 update Ran query from file 5.1.0-2024-03-28. Query text: INSERT INTO #__postinstall_messages (extension_id, title_key, `description.
2024-04-17T10:10:08+00:00 INFO 193.203.14.65 update End of SQL updates.
2024-04-17T10:10:08+00:00 INFO 193.203.14.65 update Uninstalling extensions
2024-04-17T10:10:08+00:00 INFO 193.203.14.65 update Deleting removed files and folders.
2024-04-17T10:10:09+00:00 INFO 193.203.14.65 update Cleaning up after installation.
2024-04-17T10:10:09+00:00 INFO 193.203.14.65 update Update to version 5.1.0 is complete.
alecrespi78 commented 7 months ago

To be precise, I performed a second update with the "Upload and Update" function... perhaps this log is redirected on the second attempt? Is there a way to recover the previous log?

At the end of the first attempt (online update) in the System >> Maintenance >> Database section there were inconsistencies on the "Joomla CMS" line and I used the "Update Structure" button. After that everything seemed ok (except for the errors in my initial post).

brianteeman commented 7 months ago

yes that would explain why the log shows no updates.

"Update Structure" does exactly AND only that. It will not make other changes that may be needed like additonal data records. So it would create a missing tuf database table but it would not create the missing data in that table,

SniperSister commented 7 months ago

So that would mean that the first update most likely failed before the SQL statements could be executed. The repair process created the table but not the row, resulting in the missing data.

alecrespi78 commented 7 months ago

If only the #__tuf_metadata table was involved, can I consider the issue closed? Or do I need to check some other table?

richard67 commented 7 months ago

If only the #__tuf_metadata table was involved, can I consider the issue closed? Or do I need to check some other table?

@alecrespi78 If it is right that no SQL statements were executed with the first update, then all script named like 5.1.0-*.sql which could be found here have not run: https://github.com/joomla/joomla-cms/tree/5.1-dev/administrator/components/com_admin/sql/updates/mysql .

These are the scripts

The "Fix structure" button has only run the DDL (data definition language) SQL statements, i.e. "CREATE TABLE" or "ALTER TABLE", but not the DML (data manipulation language) SQL statements like e.g. "INSERT" or "UPDATE" or "DELETE". You should check in phpMyAdmin if it needs to run these statements (i.e. check if the result of that statement is already there or not) and have to run them in phpMyAdmin.

umbcel commented 7 months ago

I have the same problem after update to 5.1. The joomla update log not contain errors but I get "0 Internal check failed." in Extensions Update and more.

An error has occurred.

0 Internal check failed.

Call Stack

# | Function | Location -- | -- | -- 1 | () | JROOT\libraries\vendor\paragonie\sodium_compat\src\Core32\Curve25519.php:1515 2 | ParagonIE_Sodium_Core32_Curve25519::ge_frombytes_negate_vartime() | JROOT\libraries\vendor\paragonie\sodium_compat\src\Core32\Ed25519.php:309 3 | ParagonIE_Sodium_Core32_Ed25519::verify_detached() | JROOT\libraries\vendor\paragonie\sodium_compat\src\Crypto32.php:1652 4 | ParagonIE_Sodium_Crypto32::sign_verify_detached() | JROOT\libraries\vendor\paragonie\sodium_compat\src\Compat.php:2898 5 | ParagonIE_Sodium_Compat::crypto_sign_verify_detached() | JROOT\libraries\vendor\paragonie\sodium_compat\lib\php72compat.php:1214 6 | sodium_crypto_sign_verify_detached() | JROOT\libraries\vendor\php-tuf\php-tuf\src\Client\SignatureVerifier.php:115 7 | Tuf\Client\SignatureVerifier->verifySingleSignature() | JROOT\libraries\vendor\php-tuf\php-tuf\src\Client\SignatureVerifier.php:68 8 | Tuf\Client\SignatureVerifier->checkSignatures() | JROOT\libraries\vendor\php-tuf\php-tuf\src\Metadata\Verifier\TargetsVerifier.php:43 9 | Tuf\Metadata\Verifier\TargetsVerifier->verify() | JROOT\libraries\vendor\php-tuf\php-tuf\src\Metadata\Verifier\UniversalVerifier.php:59 10 | Tuf\Metadata\Verifier\UniversalVerifier->verify() | JROOT\libraries\vendor\php-tuf\php-tuf\src\Client\Updater.php:395 11 | Tuf\Client\Updater->Tuf\Client\{closure}() | JROOT\libraries\vendor\guzzlehttp\promises\src\Promise.php:204 12 | GuzzleHttp\Promise\Promise::callHandler() | JROOT\libraries\vendor\guzzlehttp\promises\src\Promise.php:153 13 | GuzzleHttp\Promise\Promise::GuzzleHttp\Promise\{closure}() | JROOT\libraries\vendor\guzzlehttp\promises\src\TaskQueue.php:48 14 | GuzzleHttp\Promise\TaskQueue->run() | JROOT\libraries\vendor\guzzlehttp\promises\src\Promise.php:248 15 | GuzzleHttp\Promise\Promise->invokeWaitFn() | JROOT\libraries\vendor\guzzlehttp\promises\src\Promise.php:224 16 | GuzzleHttp\Promise\Promise->waitIfPending() | JROOT\libraries\vendor\guzzlehttp\promises\src\Promise.php:269 17 | GuzzleHttp\Promise\Promise->invokeWaitList() | JROOT\libraries\vendor\guzzlehttp\promises\src\Promise.php:226 18 | GuzzleHttp\Promise\Promise->waitIfPending() | JROOT\libraries\vendor\guzzlehttp\promises\src\Promise.php:62 19 | GuzzleHttp\Promise\Promise->wait() | JROOT\libraries\vendor\php-tuf\php-tuf\src\Client\Updater.php:169 20 | Tuf\Client\Updater->refresh() | JROOT\libraries\src\TUF\TufFetcher.php:117 21 | Joomla\CMS\TUF\TufFetcher->getValidUpdate() | JROOT\libraries\src\Updater\Adapter\TufAdapter.php:84 22 | Joomla\CMS\Updater\Adapter\TufAdapter->getUpdateTargets() | JROOT\libraries\src\Updater\Adapter\TufAdapter.php:50 23 | Joomla\CMS\Updater\Adapter\TufAdapter->findUpdate() | JROOT\libraries\src\Updater\Updater.php:263 24 | Joomla\CMS\Updater\Updater->getUpdateObjectsForSite() | JROOT\libraries\src\Updater\Updater.php:164 25 | Joomla\CMS\Updater\Updater->findUpdates() | JROOT\administrator\components\com_installer\src\Model\UpdateModel.php:275 26 | Joomla\Component\Installer\Administrator\Model\UpdateModel->findUpdates() | JROOT\administrator\components\com_installer\src\Controller\UpdateController.php:113 27 | Joomla\Component\Installer\Administrator\Controller\UpdateController->find() | JROOT\libraries\src\MVC\Controller\BaseController.php:730 28 | Joomla\CMS\MVC\Controller\BaseController->execute() | JROOT\libraries\src\Dispatcher\ComponentDispatcher.php:143 29 | Joomla\CMS\Dispatcher\ComponentDispatcher->dispatch() | JROOT\libraries\src\Component\ComponentHelper.php:361 30 | Joomla\CMS\Component\ComponentHelper::renderComponent() | JROOT\libraries\src\Application\AdministratorApplication.php:150 31 | Joomla\CMS\Application\AdministratorApplication->dispatch() | JROOT\libraries\src\Application\AdministratorApplication.php:195 32 | Joomla\CMS\Application\AdministratorApplication->doExecute() | JROOT\libraries\src\Application\CMSApplication.php:306 33 | Joomla\CMS\Application\CMSApplication->execute() | JROOT\administrator\includes\app.php:58 34 | require_once() | JROOT\administrator\index.php:32

immagine

I have this issue on all updated sites so I hope in a quick update to fix this issue.

brianteeman commented 7 months ago

please post the update log and also the method you used to perform the update!!

umbcel commented 7 months ago

Hi Brian, I updated Joomla by the automated Joomla Backend functionality and this is the update log:

#

<?php die('Forbidden.'); ?>

Date: 2024-04-17 19:22:02 UTC

Software: Joomla! 5.0.3 Stable [ Kuboresha ] 20-February-2024 16:00 GMT

Fields: datetime priority clientip category message

2024-04-17T19:22:02+00:00 INFO 213.203.154.248 update Test logging 2024-04-17T19:22:02+00:00 INFO 213.203.154.248 update Update started by user Umberto Celentano (797). Old version is 5.0.3. 2024-04-17T19:22:03+00:00 INFO 213.203.154.248 update Downloading update file from https://s3-us-west-2.amazonaws.com/joomla-official-downloads/joomladownloads/joomla5/Joomla_5.1.0-Stable-Update_Package.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA6LXDJLNUINX2AVMH%2F20240417%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20240417T192208Z&X-Amz-Expires=60&X-Amz-SignedHeaders=host&X-Amz-Signature=626da0ae032c6fe4d8d3827b030dcede9a03d53b8e6a0a20617cd4ab36683a10. 2024-04-17T19:22:07+00:00 INFO 213.203.154.248 update File Joomla_5.1.0-Stable-Update_Package.zip downloaded. 2024-04-17T19:22:07+00:00 INFO 213.203.154.248 update Starting installation of new version. 2024-04-17T19:22:47+00:00 INFO 213.203.154.248 update Finalising installation. 2024-04-17T19:22:47+00:00 INFO 213.203.154.248 update Start of SQL updates. 2024-04-17T19:22:47+00:00 INFO 213.203.154.248 update The current database version (schema) is 5.0.0-2023-09-11. 2024-04-17T19:22:48+00:00 INFO 213.203.154.248 update Ran query from file 5.1.0-2023-11-28. Query text: INSERT INTO #__extensions (package_id, name, type, element, folder, . 2024-04-17T19:22:48+00:00 INFO 213.203.154.248 update Ran query from file 5.1.0-2024-01-04. Query text: ALTER TABLE #__fields_values MODIFY value MEDIUMTEXT;. 2024-04-17T19:22:48+00:00 INFO 213.203.154.248 update Ran query from file 5.1.0-2024-02-10. Query text: ALTER TABLE #__guidedtour_steps ADD COLUMN params text NULL ;. 2024-04-17T19:22:48+00:00 INFO 213.203.154.248 update Ran query from file 5.1.0-2024-02-24. Query text: CREATE TABLE IF NOT EXISTS #__tuf_metadata ( id int NOT NULL AUTO_INCREMEN. 2024-04-17T19:22:48+00:00 INFO 213.203.154.248 update Ran query from file 5.1.0-2024-02-24. Query text: INSERT INTO #__tuf_metadata (update_site_id, root) SELECT ue.update_site_. 2024-04-17T19:22:48+00:00 INFO 213.203.154.248 update Ran query from file 5.1.0-2024-02-24. Query text: UPDATE#update_sitesSETtype= 'tuf',location= 'https://update.joo. 2024-04-17T19:22:48+00:00 INFO 213.203.154.248 update Ran query from file 5.1.0-2024-02-25. Query text: INSERT INTO#__extensions(package_id,name,type,element,folder, . 2024-04-17T19:22:48+00:00 INFO 213.203.154.248 update Ran query from file 5.1.0-2024-03-08. Query text: INSERT IGNORE INTO#guidedtour_steps(tour_id,title,published,desc. 2024-04-17T19:22:48+00:00 INFO 213.203.154.248 update Ran query from file 5.1.0-2024-03-08. Query text: ALTER TABLE #__guidedtours ADD COLUMN autostart int NOT NULL DEFAULT 0 ;. 2024-04-17T19:22:48+00:00 INFO 213.203.154.248 update Ran query from file 5.1.0-2024-03-08. Query text: INSERT IGNORE INTO #__guidedtours (title, uid, description, ordering, . 2024-04-17T19:22:48+00:00 INFO 213.203.154.248 update Ran query from file 5.1.0-2024-03-08. Query text: UPDATE #__guidedtour_steps SET tour_id = LAST_INSERT_ID() WHERE tour_id=0;. 2024-04-17T19:22:49+00:00 INFO 213.203.154.248 update Ran query from file 5.1.0-2024-03-28. Query text: INSERT INTO #__postinstall_messages (extension_id, title_key, `description. 2024-04-17T19:22:49+00:00 INFO 213.203.154.248 update End of SQL updates. 2024-04-17T19:22:49+00:00 INFO 213.203.154.248 update Uninstalling extensions 2024-04-17T19:22:49+00:00 INFO 213.203.154.248 update Deleting removed files and folders. 2024-04-17T19:22:51+00:00 INFO 213.203.154.248 update Cleaning up after installation. 2024-04-17T19:22:51+00:00 INFO 213.203.154.248 update Update to version 5.1.0 is complete.

richard67 commented 7 months ago

@umbcel

I have the same problem after update to 5.1.

No, it’s definitely not the same issue. You get a completely different error message compared to the one in the initial post of this issue.

Yours:

0 Internal check failed.

The one from this issue:

0 Could not load root metadata.

And your log shows that all SQL statements were run without error.

So maybe you should open a new issue for your case.

alecrespi78 commented 7 months ago

If only the #__tuf_metadata table was involved, can I consider the issue closed? Or do I need to check some other table?

@alecrespi78 If it is right that no SQL statements were executed with the first update, then all script named like 5.1.0-*.sql which could be found here have not run: https://github.com/joomla/joomla-cms/tree/5.1-dev/administrator/components/com_admin/sql/updates/mysql .

These are the scripts

  • administrator/components/com_admin/sql/updates/mysql/5.1.0-2023-11-28.sql
  • administrator/components/com_admin/sql/updates/mysql/5.1.0-2024-01-04.sql
  • administrator/components/com_admin/sql/updates/mysql/5.1.0-2024-02-10.sql
  • administrator/components/com_admin/sql/updates/mysql/5.1.0-2024-02-24.sql
  • administrator/components/com_admin/sql/updates/mysql/5.1.0-2024-02-25.sql
  • administrator/components/com_admin/sql/updates/mysql/5.1.0-2024-03-08.sql
  • administrator/components/com_admin/sql/updates/mysql/5.1.0-2024-03-28.sql

The "Fix structure" button has only run the DDL (data definition language) SQL statements, i.e. "CREATE TABLE" or "ALTER TABLE", but not the DML (data manipulation language) SQL statements like e.g. "INSERT" or "UPDATE" or "DELETE". You should check in phpMyAdmin if it needs to run these statements (i.e. check if the result of that statement is already there or not) and have to run them in phpMyAdmin.

Thank you, I applied them all!

SniperSister commented 7 months ago

I have this issue on all updated sites so I hope in a quick update to fix this issue.

@umbcel are all those sites hosted on the same machine or at the same hosting company? Can you post the content of the tuf_metadata table?

umbcel commented 7 months ago

I have this issue on all updated sites so I hope in a quick update to fix this issue.

@umbcel are all those sites hosted on the same machine or at the same hosting company? Can you post the content of the tuf_metadata table?

Yes, all those sites are hosted on the same machine. The contemt of the tuf_metadata table is: 1 1 {"signed":{"_type":"root","spec_version":"1.0","version":2,"expires":"2025-03-02T11:22:17Z","keys":{"07eb082f367c034a95878687f6648aa76d93652b6ee73e58817053d89af6c44f":{"keytype":"ed25519","scheme":"ed25519","keyid_hash_algorithms":["sha256","sha512"],"keyval":{"public":"9b2af2d9b9727227735253d795bd27ea8f0e294a5f3603e822dc5052b44802b9"}},"1b1b1dd55b2c1c7258714cf1c1ae06f23e4607b28c762d016a9d81c48ffe5669":{"keytype":"ed25519","scheme":"ed25519","keyid_hash_algorithms":["sha256","sha512"],"keyval":{"public":"a18e5ebabc19d5d5984b601a292ece61ba3662ab2d071dc520da5bd4f8948799"}},"2dcaf3d0e552f150792f7c636d45429246dcfa34ac35b46a44f5c87cd17d457e":{"keytype":"ed25519","scheme":"ed25519","keyid_hash_algorithms":["sha256","sha512"],"keyval":{"public":"cb0a7a131961a20edea051d6dc2b091fb650bd399bd8514adb67b3c60db9f8f9"}},"31dd7c7290d664c9b88c0dead2697175293ea7df81b7f24153a37370fd3901c3":{"keytype":"ed25519","scheme":"ed25519","keyid_hash_algorithms":["sha256","sha512"],"keyval":{"public":"589d029a68b470deff1ca16dbf3eea6b5b3fcba0ae7bb52c468abc7fb058b2a2"}},"9e41a9d62d94c6a1c8a304f62c5bd72d84a9f286f27e8327cedeacb09e5156cc":{"keytype":"ed25519","scheme":"ed25519","keyid_hash_algorithms":["sha256","sha512"],"keyval":{"public":"6043c8bacc76ac5c9750f45454dd865c6ca1fc57d69e14cc192cfd420f6a66a9"}}},"roles":{"root":{"keyids":["1b1b1dd55b2c1c7258714cf1c1ae06f23e4607b28c762d016a9d81c48ffe5669","2dcaf3d0e552f150792f7c636d45429246dcfa34ac35b46a44f5c87cd17d457e"],"threshold":1},"snapshot":{"keyids":["07eb082f367c034a95878687f6648aa76d93652b6ee73e58817053d89af6c44f","2dcaf3d0e552f150792f7c636d45429246dcfa34ac35b46a44f5c87cd17d457e"],"threshold":1},"targets":{"keyids":["31dd7c7290d664c9b88c0dead2697175293ea7df81b7f24153a37370fd3901c3"],"threshold":1},"timestamp":{"keyids":["9e41a9d62d94c6a1c8a304f62c5bd72d84a9f286f27e8327cedeacb09e5156cc"],"threshold":1}},"consistent_snapshot":true},"signatures":[{"keyid":"2dcaf3d0e552f150792f7c636d45429246dcfa34ac35b46a44f5c87cd17d457e","sig":"2a225a560ec0837b721d4c5e379fedbd3c7c9079a94e6b31e47e0184c8b95421b6036b4286c5d90f29ab4c468d79a712fdb65e96511394ceb3aa8e2b3983a501"},{"keyid":"1b1b1dd55b2c1c7258714cf1c1ae06f23e4607b28c762d016a9d81c48ffe5669","sig":"8ce0b2a7bdc1e6dcba12081f440510df0a593c072dcf591631c2dd0f456844a7da63be8e8ac31ffbddf42641fde84dc733a336031d182c2163b4c1eaf2117005"}]}

SniperSister commented 7 months ago

@umbcel can you please try to reproduce the issue with the same site but on a different machine at a different host? The error seems to be related to the handling of the cryto logic in an upstream library and I assume that some sort of environment issue (OS, PHP Version etc) plays a role. Would like to confirm that assumption.

umbcel commented 7 months ago

@umbcel can you please try to reproduce the issue with the same site but on a different machine at a different host? The error seems to be related to the handling of the cryto logic in an upstream library and I assume that some sort of environment issue (OS, PHP Version etc) plays a role. Would like to confirm that assumption.

I can't. I have all those web sites on my machine, my hosting, and no problem happen with older 3.x/4.x/5.x Joomla releases. This problem started after I updated web sites to Joomla 5.1.

SniperSister commented 7 months ago

Well that's a pity. Could you please at least provide as many environment informations as possible? OS, PHP info() output, CPU?

umbcel commented 7 months ago

Well that's a pity. Could you please at least provide as many environment informations as possible? OS, PHP info() output, CPU?

I prefer not share system info in public environment for security reason. I tried also to install a fresh Joomla 5.1 and the problem happen again. immagine

SniperSister commented 7 months ago

I tried also to install a fresh Joomla 5.1 and the problem happen again.

... which supports the assumption that the problem is related to the environment.

umbcel commented 7 months ago

I tried also to install a fresh Joomla 5.1 and the problem happen again.

... which supports the assumption that the problem is related to the environment.

So why was everything working fine just before installing the updates? Why did the previous version of Joomla 5 work well?

SniperSister commented 7 months ago

So why was everything working fine just before installing the updates? Why did the previous version of Joomla 5 work well?

Because Joomla 5.1 introduced a new function that ensures that an update, that is fetched from the joomla server, is actually a legit one and not one that has been manipulated by an attacker.

The error message is having its source in the cryptographic verification process which works fine on a wide range of different machines. In fact, you are the only person with that issue so far - so something about that environment has to be special.

umbcel commented 7 months ago

So why was everything working fine just before installing the updates? Why did the previous version of Joomla 5 work well?

Because Joomla 5.1 introduced a new function that ensures that an update, that is fetched from the joomla server, is actually a legit one and not one that has been manipulated by an attacker.

The error message is having it's source in the cryptographic verification process which works fine on a wide range of different machines. In fact, you are the only person with that issue so far - so something about that environment has to be special.

Ok, thanks, this is a more logical explanation and is likely to be the case. Where can I find information on these changes to remedy them on my hosting?

umbcel commented 7 months ago

What I need to check and implement on my hosting to avoid these problems?

SniperSister commented 7 months ago

Where can I find information on these changes to remedy them on my hosting?

Joomla has implemented TUF as its updating framework https://theupdateframework.io/

We are using PHP-TUF as the implementation within Joomla to consume the updates: https://github.com/joomla-backports/php-tuf

And PHP-TUF uses the compat package for lib_sodium to do the crypto operations: https://github.com/paragonie/sodium_compat

The error message occurs in the last part of the chain, the crypto library.

What I need to check and implement on my hosting to avoid these problems?

That's a though one. At a first step, check if your PHP native sodium support: https://www.php.net/manual/en/sodium.installation.php and if not try to enable it. If that solves the issue, it's most likely a bug in the upstream library that needs further debugging.

umbcel commented 7 months ago

Yeahhhh! Problem solved! I just enabled Sodium support in PHP and all run fine now. Thank you very much!

SniperSister commented 7 months ago

Interesting. Can you at least provide the used PHP version and operation system so I can have same basic information to look for the root cause in the upstream library?

umbcel commented 7 months ago

Interesting. Can you at least provide the used PHP version and operation system so I can have same basic information to look for the root cause in the upstream library?

Windows system Apache 2.4 PHP 8.2 Custom config

ThreeCatsMarketing commented 6 months ago

I'm getting these errors as well. My environment is Apache 2.4, PHP 8.3.6. I'm running Joomla! 5.1.0. Here's the stack trace:

2024-05-19T20:45:45+00:00 CRITICAL 45.17.166.59 error Uncaught Throwable of type JsonException thrown with message "Syntax error". Stack trace: #0 [ROOT]/libraries/vendor/php-tuf/php-tuf/src/Metadata/MetadataBase.php(90): json_decode('', true, 512, 4194304) https://github.com/akeeba/panopticon/pull/1 [ROOT]/libraries/vendor/php-tuf/php-tuf/src/Client/Repository.php(67): Tuf\Metadata\MetadataBase::createFromJson('') https://github.com/akeeba/panopticon/pull/2 [ROOT]/libraries/vendor/php-tuf/php-tuf/src/Client/Updater.php(181): Tuf\Client\Repository->getTimestamp() https://github.com/akeeba/panopticon/pull/3 [ROOT]/libraries/vendor/php-tuf/php-tuf/src/Client/Updater.php(148): Tuf\Client\Updater->updateTimestamp() https://github.com/akeeba/panopticon/issues/4 [ROOT]/libraries/src/TUF/TufFetcher.php(117): Tuf\Client\Updater->refresh() https://github.com/akeeba/panopticon/issues/5 [ROOT]/libraries/src/Updater/Adapter/TufAdapter.php(84): Joomla\CMS\TUF\TufFetcher->getValidUpdate() https://github.com/akeeba/panopticon/issues/6 [ROOT]/libraries/src/Updater/Adapter/TufAdapter.php(50): Joomla\CMS\Updater\Adapter\TufAdapter->getUpdateTargets(Array) https://github.com/akeeba/panopticon/issues/7 [ROOT]/libraries/src/Updater/Updater.php(263): Joomla\CMS\Updater\Adapter\TufAdapter->findUpdate(Array) https://github.com/akeeba/panopticon/issues/8 [ROOT]/libraries/src/Updater/Updater.php(164): Joomla\CMS\Updater\Updater->getUpdateObjectsForSite(Array, 4, false) https://github.com/akeeba/panopticon/issues/9 [ROOT]/administrator/components/com_installer/src/Model/UpdateModel.php(275): Joomla\CMS\Updater\Updater->findUpdates(0, 3600, 4) https://github.com/akeeba/panopticon/issues/10 [ROOT]/administrator/components/com_installer/src/Controller/UpdateController.php(161): Joomla\Component\Installer\Administrator\Model\UpdateModel->findUpdates(0, 3600, 4) https://github.com/akeeba/panopticon/issues/11 [ROOT]/libraries/src/MVC/Controller/BaseController.php(730): Joomla\Component\Installer\Administrator\Controller\UpdateController->ajax() https://github.com/akeeba/panopticon/issues/12 [ROOT]/libraries/src/Dispatcher/ComponentDispatcher.php(143): Joomla\CMS\MVC\Controller\BaseController->execute('ajax') https://github.com/akeeba/panopticon/issues/13 [ROOT]/libraries/src/Component/ComponentHelper.php(361): Joomla\CMS\Dispatcher\ComponentDispatcher->dispatch() https://github.com/akeeba/panopticon/issues/14 [ROOT]/libraries/src/Application/AdministratorApplication.php(150): Joomla\CMS\Component\ComponentHelper::renderComponent('com_installer') https://github.com/akeeba/panopticon/issues/15 [ROOT]/libraries/src/Application/AdministratorApplication.php(195): Joomla\CMS\Application\AdministratorApplication->dispatch() https://github.com/akeeba/panopticon/issues/16 [ROOT]/libraries/src/Application/CMSApplication.php(306): Joomla\CMS\Application\AdministratorApplication->doExecute() https://github.com/akeeba/panopticon/issues/17 [ROOT]/administrator/includes/app.php(58): Joomla\CMS\Application\CMSApplication->execute() https://github.com/akeeba/panopticon/issues/18 [ROOT]/administrator/index.php(32): require_once('/home/amybitcov...') https://github.com/akeeba/panopticon/issues/19 {main}

csbalazs commented 5 months ago

I have the same error. I clicked on it and the 5.1.1 update appeared. After that I get a Could not load root metadata error. After that I just get a red X to update. I think this error should be handled if the tuf_metadata table is empty somewhere.

imperpl commented 5 months ago

To solve - install second instance of joomla on the server, copy using phpmyadmin or similiar table "updates" and move it to the main instance (which you would like to fix)

csbalazs commented 5 months ago

When TUF was introduced, the update should have checked if there was such a table (tuf_metadata) and if it contained the correct data. If not, it should have been fixed automatically.

SniperSister commented 5 months ago

When TUF was introduced

The table and the correct data was created on the update to 5.1.0, if however, for whatever reason, that update wasn't performed correctly, you'll end up with exactly this problem. So the root issue is not the introduction of TUF or the lack of logic to add the table, but an incomplete update.

csbalazs commented 5 months ago

Let's look at it from another perspective. There is a 5.1.0 Joomla! page, where the tuf_metadata table exists, but there is no data in it. According to the database maintainer: All database table structures are up to date. After that, if I click on the 5.1.1 update and get a Could not load root metadata error, it is a feature and not a bug.

tristan-bellosta commented 5 months ago

I had the same problem on a developing website. I don't remember having an incomplete update, but it's possible.

Enabling sodium extension didn't help in my case.

To get around this, I temporarily modified the libraries/src/Updater/Updater.php file line 263: $update_result = $adaptor->findUpdate($updateSite); which I transformed into $update_result = false;

This allowed me to access the com_joomlaupdate component, and I then imported the latest Joomla update pack. After installation everything was OK.

csbalazs commented 5 months ago

@tristan-bellosta, all you need is the following: Install a new Joomla, then export the contents of the tuf_metadata table, and finally import this into the database of your existing Joomla site. Note: The problem remains that Joomla! according to the empty tuf_metadata table does not represent a database problem. Ergo, it is not possible to fix the error automatically by fixing the database.