Closed richard67 closed 4 years ago
Confirmed. Set to Beta blocker.
@infograf768 Please confirm and paste your system information.
@alikon Please tested exactly as described, with localized multilingual data, if it is really not an issue when using PostgreSQL (PDO).
So MySQL 5.7, like in my case, too.
Also note these 2 errors when updating [09-Apr-2020 08:08:34 UTC] PHP Notice: Undefined property: Joomla\CMS\Table\Extension::$locked in /Applications/MAMP/htdocs/joomla-cms-3.10-dev/libraries/src/Installer/InstallerAdapter.php on line 1155 [09-Apr-2020 08:08:35 UTC] PHP Warning: count(): Parameter must be an array or an object that implements Countable in /Applications/MAMP/htdocs/joomla-cms-3.10-dev/plugins/installer/override/override.php on line 134
1st one comes from #28462 , 2nd one could be older.
But these are separate issues.
confirm the warning about plugins/installer/override @richard67 confirm tested as described
@richard67 confirm tested as described
Do you mean you can confirm the issue with PostgreSQL, too? Or do you mean you can confirm what I wrote, that the issue is not there with PostgreSQL? It is not clear to me from your comment.
i confirm that the issue is not there with PostgreSQL (PDO)
With debug switched on in global config:
But if it's this, why doesn't it happen with any other db driver than MySQLi? Am maximum confused now.
Ah there is another exception in the stack:
Joomla\Database\Exception\PrepareStatementFailureException: Unknown column 'm.publish_up' in 'where clause' at /home/richard/lamp/public_html/joomla-cms-3.10-dev/libraries/vendor/joomla/database/src/Mysqli/MysqliStatement.php:141 at Joomla\Database\Mysqli\MysqliStatement->__construct(object(mysqli), 'SELECT
m
.id
,m
.menutype
,m
.title
,m
.alias
,m
.note
,m
.link
,m
.type
,m
.level
,m
.language
,m
.browserNav
,m
.access
,m
.params
,m
.home
,m
.img
,m
.template_style_id
,m
.component_id
,m
.parent_id
,m
.path
ASroute
,e
.element
AScomponent
FROMj3ux0_menu
ASm
LEFT JOINj3ux0_extensions
ASe
ONm
.component_id
=e
.extension_id
WHERE ((m
.published
= 1 ANDm
.parent_id
> 0 ANDm
.client_id
= 0) AND (m
.publish_up
IS NULL ORm
.publish_up
<= ?)) AND (m
.publish_down
IS NULL ORm
.publish_down
>= ?)ORDER BYm
.lft
') (/home/richard/lamp/public_html/joomla-cms-3.10-dev/libraries/vendor/joomla/database/src/Mysqli/MysqliDriver.php:1048) at Joomla\Database\Mysqli\MysqliDriver->prepareStatement('SELECTm
.id
,m
.menutype
,m
.title
,m
.alias
,m
.note
,m
.link
,m
.type
,m
.level
,m
.language
,m
.browserNav
,m
.access
,m
.params
,m
.home
,m
.img
,m
.template_style_id
,m
.component_id
,m
.parent_id
,m
.path
ASroute
,e
.element
AScomponent
FROMj3ux0_menu
ASm
LEFT JOINj3ux0_extensions
ASe
ONm
.component_id
=e
.extension_id
WHERE ((m
.published
= 1 ANDm
.parent_id
> 0 ANDm
.client_id
= 0) AND (m
.publish_up
IS NULL ORm
.publish_up
<= :currentDate1)) AND (m
.publish_down
IS NULL ORm
.publish_down
>= :currentDate2)ORDER BYm
.lft
') (/home/richard/lamp/public_html/joomla-cms-3.10-dev/libraries/vendor/joomla/database/src/DatabaseDriver.php:1891) at Joomla\Database\DatabaseDriver->setQuery(object(MysqliQuery)) (/home/richard/lamp/public_html/joomla-cms-3.10-dev/libraries/src/Menu/SiteMenu.php:155) at Joomla\CMS\Menu\SiteMenu->Joomla\CMS\Menu{closure}() ....
Prepared statement ... that fits better to db stuff.
Meanwhile I think the above posts just show secondary errors but not the root cause. Of course prepared statements will fail in some PHP running after the database update when the database update hasn't run any SQL statement to add the publish up and down columns to the menu table.
Can be also reproduced when using French only, without Persian.
Let’s also test by not adding localised content, i.e with only the languagefilter plugin and with/without the langswitcher module. Can’t right now because I am on Ipad.
@infograf768 sure, test it. I think I have tested that before, and the issue did not happen, but to be sure test again because I am not 100% sure now anymore.
At the end it will turn out that it's only happening when using such exotic languages like French, but not with normal languages like Chinese, Farsi, Arab, Klingon ... so maybe we just drop the support for French in J4 .. shall they learn English or German ;-) (teasing jokes)
I did a database export before and after the update and there was no difference in structure or content coming form any of the update sql scripts ... only the update site info and so on, available updates, last login (because I had logged out in between), only such stuff has changes.
@infograf768 I've just tested with installing French and enabling the multilanguage function and disabling localized content. Same result as with localized content. But I am sure from past tests when I did a monolanguage installation and after that logged in to the backend, installed languages and published content languages and enabled the language filter, then all was fine. But not 100% sure.
@infograf768 Interesting: Now I have done everything to reproduce the issue, i.e. made multilingual installation with French and enabled multilanguage feature and enabled localized content. But before I did the Joomla Update, I deactivated the language filter plugin. And now it worked!!! Does that help you with having an idea what could go wrong where?
Or is maybe something with that saving or resetting the session? => @wilsonge ?
Figured out the issue. So when we have the language filter plugin enabled we always load the site menu to attach the build rules https://github.com/joomla/joomla-cms/blob/4.0-dev/plugins/system/languagefilter/languagefilter.php#L158
The problem is when we hit the login prompt we haven't run any of the SQL yet (this is normally the case if you check a Joomla update when you log in after performing the file update and is why you get the invalid template message).
The issue is that loading the site menu preloads all the items in the site menu. Normally this isn't a major issue. However we come to one of the new J4 features: Allowing publish up/down of menu items. https://github.com/joomla/joomla-cms/blob/4.0-dev/administrator/components/com_admin/sql/updates/mysql/4.0.0-2019-03-30.sql
The problem here is that we are now loading the site menu items (including the publish up/down columns before the column has been created!)
This leads to the exception that @richard67 mentioned here https://github.com/joomla/joomla-cms/issues/28614#issuecomment-611539235
The fact that postgres doesn't error when you try and filter by a non-existent column is interesting (and I'd claim a bug in postgres :) )
@wilsonge Any idea yet how we shall solve it? And who shall do it? I am available and would work on it if nobody else does.
@wilsonge With MySQL (PDO) it's as strange as with PostgreSQL (PDO), so it seems to be a PDO issue that there is no problem with missing columns.
@wilsonge Would it help to add these columns to the menu table with an update sql in 3.10 so we can assume we have them when updating to 4, assuming that anyone who updates a 3.9 or earlier with Upload&Update is doing that on own risk?
Update: Alternatively we could add them in PHP in the update procedure before that login step which makes it fail then?
FWIW @infograf768 i don't think this is specific to j4 - it would happen in j3 if we did any sort of change to the menu table :(
Try this one https://github.com/joomla/joomla-cms/pull/28635 - this also has the advantage that in the majority of the administrator client after patch in j4 you should see at least 1 less query because we no longer load all the site menu items unless we actually start to use the site router to build links.
please test #28635
Steps to reproduce the issue
Use a MySQL or MariaDB database with the "MySQLi" driver, i.e. not the "MySQL (PDO)" driver.
Install a current 3.10 nightly or 3.10-dev branch.
At the end of the installation, chose the additonal step for installing languages and install French and Persian.
When languages have been installed, chose to enable multilanguage and select to install localized content.
Then finish the installation and login to backend.
Go to the Joomla Update component and update to last 4.0 nightly build, either using the Live Update with following custom URL https://update.joomla.org/core/nightlies/next_major_list.xml or the Upload & Update with the previously downloaded update package from here https://developer.joomla.org/nightlies/Joomla_4.0.0-beta1-dev-Development-Update_Package.zip.
Start the update and enter credentials whenever asked for it.
Expected result
Update successful.
Actual result
The update starts to unpack the update package.
Then immediately after unpacking of the package:
In
administrator/logs/joomla_update.php
:I.e. it fails before the first SQL statement of any update SQL script is executed or with the very first statement. Maybe a problem with connecting?
The log of the database server shows nothing,
System information (as much as possible)
PHP 7.2, MySQL 5.7, Apache 2.4 on Ubuntu.
Additional comments
I happens too when updating a staging to 4.0 nightly (for which Upload&Update has to be used because the Live Update will not find an update for 3.9 to 4). So it seems not to be a 3.10-dev issue which has been fixed already in staging.
It does not fail when no localized content is installed, i.e. only the languages are installed but no content.
It does not fail when using the "MySQL (PDO)" driver.
It does not fail when using the "PostgreSQL (PDO)" driver with a PostgreSQL database.
I don't know when it has stopped working. I have tested many updates here in recent weeks, but not with multilingual sites having language-specific content. Thanks to @infograf768 who first discovered this issue today.