[RFC] Do not distribute core frontend templates in update release packages

[PhilETaylor]

I propose that future versions of Joomla do not distribute core frontend templates in UPDATE release packages

Its best practice to only leave files in your webspace that are required for the functioning of the site, having additional templates installed, can lead to unexpected results (see recent beez acl/overide misconfiguration for example).

Maybe even move the templates to their own release/update channel? This fits with the (ancient) roadmap of "Light Core and Core Supported Extensions" The templates could become decoupled very quickly and easy to implement this.

Thoughts?

Steps to reproduce the issue

Upgrade to the next Joomla version, having deleted the core templates, and using a custom template

Expected result

It is expected that no new templates would be added (readded) by Joomla Upgrade

Actual result

All core Joomla templates are re-added to the servers file system in /templates, making people who have cleaned up their site upset (even angry) that they have to then go and delete these fluff files again from their file system

[zero-24]

E.g the JSST fixed a sec bug in beez so you want this unpatched in the future? What about layout changes or fixes we implement in the templates?

Beez and protostar are core joomla templates which should not be deleted (the same as the core extensions which should never be removed (you can disable them for sure ;)

[infograf768]

People should understand that they should always create a copy of the core templates if they want to modify it.

[PhilETaylor]

Beez and protostar are core joomla templates which should not be deleted

Absolute rubbish. The best practice is to remove anything from a webspace that is not required or not needed for the site to run.

the same as the core extensions which should never be removed

Actually that was the direction of Joomla! E.g. Weblinks!

so you want this unpatched in the future?

Did I say that - No.

Core templates can be extracted from the Joomla core and be distributed in their own right the same as weblinks the the updater... this will mean that security fixes in core templates could actually be FASTER distributed

@infograf768 People should understand that they should always create a copy of the core templates if they want to modify it.

Proves you never bothered reading the opening text I wrote. This is not about overwriting someones changes if they chose to use the core templates, this is about a site that has its own custom template named whatever, and doesnt want to have the Joomla templates on their site, in their webspace at all.

[infograf768]

correct, I just read the title ;)

[mbabker]

This is not practical given the way Joomla is packaged and distributed. Also, the current leadership has elected that decoupling extensions is too difficult to manage so any thoughts about removing the templates from the repository here and placing them in a standalone spot are out of the question.

No extension that is part of the core Joomla package (the new install) can be safely removed. Removing components more often than not will cause upgrade issues later, the other extension types will end up back in your filesystem even if they're not wanted.

This is not just an issue with the templates. It applies to every extension in the core package. I have a site at work that is down to 27 enabled extensions (not including the protected ones) and of this 12 are from the core package. Add this to the 58 protected extensions, and I've still disabled half of what ships with core with no way to uninstall this cruft and not have it reappear on my site. The only way to fix it is to rewrite the entire update routine to not only make it aware of the local operating environment but also the active Joomla installation (restore.php has to be aware of what extensions are installed and instead of just extracting the ZIP package check each directory (and presumably the database) to determine a core extension's status. This couples the currently standalone updater to the core Joomla application somehow (either it's going to need to read the database or share a session). Given the project's current state and historically not having a strong developer base with the skillset to write that mechanism, and the general reluctance from a lot of testers to test deep rooted architectural changes (not blaming them, just saying most stay at their skill level), the odds of something like that ever happening are slim to none.

[PhilETaylor]

the current leadership has elected that decoupling extensions is too difficult to manage

I thought that was the future of Joomla... https://developer.joomla.org/cms/roadmap.html LMAO - forked an issue about that here (https://github.com/joomla/joomla-websites/issues/807)

I think it would be a million times easier to remove TEMPLATES from Joomla, than it would be to decouple extensions. I still see no reason that the core frontend templates cannot be excluded from the patch update packages

[dgrammatiko]

restore.php has to be aware of what extensions are installed and instead of just extracting the ZIP package check each directory (and presumably the database)

I am willing to test such a PR, if that's any helpful 😉

[mbabker]

I think it would be a million times easier to remove TEMPLATES from Joomla, than it would be to decouple extensions. I still see no reason that the core frontend templates cannot be excluded from the patch update packages

Templates are extensions, and without a decoupled release process this is not possible. Leadership has decided decoupling is too hard, even our one extension which does support standalone releases the leadership elected to not issue a release for when a patch was merged to fix Windows compatibility issues with upgrades.

As much as I'd like to see it happen, I don't see it happening before Joomla 10.0 (codename never).

[Bakual]

The problem imho isn't so much to decouple things, but to find people who are willing to manage those decoupled extensions (managing PRs, doing the releases, making sure translations are done, ....),

[tonypartridge]

It might sound daft, but can we not just add a simple JFolder exists check on core templates, modules and components that can be uninstalled before moving the files? I appreciate this will add a bit of overhead, but it would avoid this issue users face. I too have the same problem when removing things that are just not needed.

[mbabker]

No. The updates are processed in a standalone file that is not coupled to the Joomla API in any way (it's not used at all except for the cleanup operation that runs after the filesystem updates have been made). During an update process, you cannot and should not load any file into memory that is being replaced during the update process otherwise you will run into scenarios where the update will break catastrophically (it's why you can't update Joomla anymore in the Extension Manager). So the update process (even if it stops using Nic's code) should NEVER be coupled to the Joomla API (no filesystem operations, no database queries, you can't even use the session started by the admin app).

Next, I'm not 100% sure if you can do a conditional "if directory exists then extract this part of the compressed package" step. Meaning you would have to extract the entire update package into the tmp directory then move it element by element into the production filesystem. That adds a LOT of overhead to the update process.

Best practice is to NOT uninstall anything that comes with the core package. Until a point that the project matures in a way this status quo changes (I doubt it'll ever happen), stop uninstalling the core extensions and simply leave them disabled.

[tonypartridge]

And there you have it, I clearly have no clue how the update api works :-).

Yep that's fine by me, but I totally get Phills point, if you can uninstall something it shouldn't be reinstalled.

On Wed, 21 Dec 2016 at 18:56, Michael Babker notifications@github.com wrote:

No. The updates are processed in a standalone file that is not coupled to the Joomla API in any way (it's not used at all except for the cleanup operation that runs after the filesystem updates have been made). During an update process, you cannot and should not load any file into memory that is being replaced during the update process otherwise you will run into scenarios where the update will break catastrophically (it's why you can't update Joomla anymore in the Extension Manager). So the update process (even if it stops using Nic's code) should NEVER be coupled to the Joomla API (no filesystem operations, no database queries, you can't even use the session started by the admin app).

Next, I'm not 100% sure if you can do a conditional "if directory exists then extract this part of the compressed package" step. Meaning you would have to extract the entire update package into the tmp directory then move it element by element into the production filesystem. That adds a LOT of overhead to the update process.

Best practice is to NOT uninstall anything that comes with the core package. Until a point that the project matures in a way this status quo changes (I doubt it'll ever happen), stop uninstalling the core extensions and simply leave them disabled.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/joomla/joomla-cms/issues/13220#issuecomment-268609260, or mute the thread https://github.com/notifications/unsubscribe-auth/ABVglvo8-91vlP2HZdEUKQxQaJPqvXX5ks5rKXZJgaJpZM4LN0lJ .

[mbabker]

I get what Phil and others want (I'd like it too). But I honestly just don't see it happening the way Joomla is managed. I hacked together an update system modeled very loosely on how both Joomla and WordPress process the filesystem operations (with the same explicit instructions about keeping it decoupled to avoid problems) and as soon as my connections with that group were severed they threw that advice out the window and last I looked they had a lot of "well how do we fix this" issues before one of their big releases. I have as much worry about that happening in Joomla and honestly we'd be back to another 3.5 development cycle where months are being spent trying to address a "oh crap we FUBAR'd the update system royally" problem, because it WILL happen.

[PhilETaylor]

Closing this as there seems to be no appetite for it.

I maintain and still advocate that it's best practice to remove anything from your web space that you do not require for the correct running of your site.