Maintain a usable Administrator interface on system plugin errors

[mavrosxristoforos]

Is your feature request related to a problem? Please describe.

Joomla has had a stable administrator interface for a very long time. It has always been almost impossible to break the Joomla back-end, unless you create a System plugin that somehow fails in the administrator. Yet, now in Joomla 4, this is suddenly quite more common, because of a lot of plugins that use the Application::isSite() and Application::isAdmin() functions that have been removed from the CMSApplication class. Interestingly, this is also one very common reason that a Joomla 3 to 4 migration may fail. When we cripple a Joomla 3 to 4 migration, or the Joomla 4 administrator interface, we lose a lot of Joomla users for no reason. That is not smart.

Describe the solution you'd like

Gracefully handle system plugin errors, at least in the back-end. Since these two functions (isAdmin and isSite) have been so widely used, why are they now treated like a sin? We could re-introduce them in the Joomla CMSApplication, and IF WE MUST, enqueueMessage a deprecated warning, so that admins know that their plugins are out-dated, but STILL be able to use their back-end to disable them.

Additional context

This is much smoother than tearing down the whole administrator interface for a couple of simple functions that could be offered in the CMSApplication class and end the story. In general, I am developing extensions for Joomla since 2008 and backwards compatibility has always been the problem. Please stop driving people away. We need to focus more on the total UX in the real world, where a Joomla site works together with its extensions, some of which may not be updated quite so often.

[crystalenka]

Γεια σου!

Thanks for opening the issue. There are indeed places that UX can be improved, so I appreciate you bringing this up!

There is a renewed interest and commitment to UX in Joomla with the creation of the Experience Team, so while we have a lot to do, you should start to see more consistent UX improvements soon.

That being said, we can't be everywhere, so please do continue to raise UX issues with the understanding that we are a small team (and always open to new contributors 😉)

Thank you!

[mavrosxristoforos]

Γεια σου Crystal! Thanks for your input. I'm glad to learn about the Experience team. I think that maintaining a functional admin interface must be made a priority, especially when talking about simple backwards compatibility.

[brianteeman]

I absolutely agree with you. I raised this multiple times but sadly with no success. It is why I created this video explainer with over 5000 hits

https://youtu.be/ZO9nmdsLlFw

[brianteeman]

Previous discussion https://github.com/joomla/joomla-cms/issues/35380

[mavrosxristoforos]

Hi Brian, I'm glad to finally talk to you. I've been wanting to for a long time. I appreciate your response. Please, let me bring in some context.

I create Joomla extensions for a living. All of my own extensions are ready for Joomla 4 and as discussed in #35380, I'm also aware that this may help separate the old stuff. I have read the Joomla documentation and backwards compatibility docs hundreds of times. I have been reading it ever since the com_hello component had comments in Brazilian Portuguese. Myself, I am banned from writing in the Joomla docs, because when I first wanted to put my 2 cents, I didn't have time to read the whole process of registering, so the Abuse filter cut me off. For life. I haven't found ANY WAY to fix that since then, so I just can't contribute my knowledge. I just wanted to say that us, developers, don't have the time and money to re-build our extensions from scratch every time that Joomla delivers a new version. Most of the time, I directly read Joomla's source code to find the expected behavior on building something. Example: Using Joomla Ajax in templates. The dev team has done it. There are no docs. I know how to do it. I can't write it.

I'm currently working on upgrading a LARGE e-Commerce website that makes use of external APIs that come with paid Joomla extensions. I am not talking about freebies here. We have at least 5 PROFESSIONAL Joomla plugins that break the Joomla back-end. This is not a major functionality that is missing. It's just a couple of placeholder functions that ease the transition.

People are leaving Joomla every day, just because of nonsense things like this. Myself, I have supported the use of Joomla for years, but I just can't live off it anymore, since the people using it are draining. I have to find a new job, or even worse: use WordPress 😄

[brianteeman]

as you can see in #35380 the code was marked as deprecated about 7-8 years ago,. So more than long enough for people to update their code. And even if they didnt the video shows you in 2 minutes how to fix that.

[mavrosxristoforos]

And that only further proves my point. If the dev team deprecated something in 2014 and we're discussing serious migration issues with it in 2022, this simply highlights the fact that core changes are not effectively made known to extension developers. Just google any Joomla class and you'll probably get OLD (sometimes even 1.5) bare API docs. No code examples, no best practices. I'm just pointing out that 1) breaking the Joomla back-end is a bad, bad, BAD solution for an already problematic communication between the core team and extension developers, and 2) we shouldn't forget that Joomla only still exists because people develop extensions for it.

[brianteeman]

and with that comment which shifts all the blame onto a few hard working VOLUNTEERS and says all the success is because of paid extension developers I am out of here.

It started off productive but now you are just plain daft.

Bye

[mavrosxristoforos]

Please don't get me wrong. I'm not saying it's the volunteers to blame. Most volunteers ARE also extension developers anyway. As I said, I want to help, but can't. Maybe others, too. That's why I raised this issue here.

Despite our differences in point of view, let's just stick to the point. Letting the Joomla back-end break is not the solution. We need to offer admins a way to fix this.

[chmst]

We need to offer admins a way to fix this.

I agree. Please make a PR and everyone will be happy.

[zero-24]

I haven't found ANY WAY to fix that since then, so I just can't contribute my knowledge

try to reach out to the docs team? now days we do also have some dev docs on GitHub: https://github.com/joomla/Manual we would love to get your help there.

I'm currently working on upgrading a LARGE e-Commerce website that makes use of external APIs that come with paid Joomla extensions. I am not talking about freebies here. We have at least 5 PROFESSIONAL Joomla plugins that break the Joomla back-end. This is not a major functionality that is missing. It's just a couple of placeholder functions that ease the transition.

Why did you not contacted the original dev when the pre Upgrade checker reported this to be not compatible? This is one of the reason for this checks, that the backend and upgrade process itself does not break. When that professional extension is not maintained any more for J4 i would go and fork it and do the search and replace or better find an alternative extension. Nothing is worse than a not maintained professional extension used on a E-Commerce site ;⁠-⁠) You still have until September 2023 until this time 3.10 gets Security patches so no hurry now.

People are leaving Joomla every day, just because of nonsense things like this. Myself, I have supported the use of Joomla for years, but I just can't live off it anymore, since the people using it are draining.

And people are comming back because new real major releases that are allowed to break b/c and do have to else we are going into a maintainability hell. You are developer too so you might know the pain of legacy code too?

I have to find a new job, or even worse: use WordPress 😄

We both know that you will not find better code there but good luck with your new job. I would recommend Drupal or Typo3 they are also great CMS, choose what is the best for your usecase maybe a clean site based on a php Framework works good too. All of that have is own advantages and disadvantages of course :⁠-⁠)

[mavrosxristoforos]

Hello Christiane! I will try to give it a shot.

Dear Tobias, I'm excited to talk to you, too! I didn't mean to make this issue so personal. Please strike, but hear me first.

try to reach out to the docs team?

I haven't found a way to do that if you get blocked by the abuse filter on your first edit. Can't send an email if your email is not confirmed yet, and can't even contact the admins through the wiki User:talk page. I'm blocked out. Please don't get me wrong. I'm not complaining. I'm just generally saying that the Joomla ecosystem needs to be more welcoming to newcomers.

Why did you not contacted the original dev when the pre Upgrade checker reported this to be not compatible?

I have updated the website to Joomla 4. I didn't break it. I know the process. I know how to do the 2-min fix that Brian mentioned, and I even know how to temporarily edit the AdministratorApplication class to include the two missing functions and test out the extensions. I am just saying that this is happening to hundreds of other people who are not like me and you.

i would go and fork it and do the search and replace or better find an alternative extension

We are forced to use some extensions because there is no alternate, and the client doesn't want to pay for building something that already exists. Some of these extensions are CLOSED source, using ioncube loader and cannot be forked, which I personally dislike.

You still have until September 2023 until this time 3.10 gets Security patches so no hurry now.

Our clients see a threatening message (for them) to update to Joomla 4, yet their admin (myself) tells them that they can't do it yet, because of some extensions. Do you know what they truly hear instead? Your site is getting old and I cannot fix it.

And people are comming back because new real major releases that are allowed to break b/c and do have to else we are going into a maintainability hell. You are developer too so you might know the pain of legacy code too?

Yes, you are right. Brian is also right. It's not the volunteers' fault. I would just like to see Joomla handling this in a smarter way that penalizes the outdated extensions, not the end-users. That's why I opened this issue. I think this is important.

We both know that you will not find better code there

That's why I'm one of the few extension devs that's still ONLY working with Joomla. In the past, I've worked with WP, Drupal, Presta, CMSMS and others, but I chose Joomla. That's why I made this 8 minute video about 10 points that Joomla is better. https://youtu.be/fo2Oq_e0ETQ

Anyway, I will try to make a PR that addresses this in a simple way: When a plugin breaks the back-end, identify the culprit plugin and offer a button to disable it, directly. Do you think this could work?

[zero-24]

I didn't mean to make this issue so personal. Please strike, but hear me first.

I'm sorry it was not meant to be personal, its just that we get so many complaints where people ignore our warnings and messages we put in there for a reason and beeing charged like "you damage our bussiness" and you "volunteers are the ideots that my extension which i have not maintained since ages break now". This does not seem to be true for you are you are looking into a constructive solution. Thats great :)

I haven't found a way to do that if you get blocked by the abuse filter on your first edit. Can't send an email if your email is not confirmed yet, and can't even contact the admins through the wiki User:talk page. I'm blocked out. Please don't get me wrong. I'm not complaining. I'm just generally saying that the Joomla ecosystem needs to be more welcoming to newcomers.

At the bottem of every joomla site like joomla.org you get a URL to: joomla/joomla-websites also on the Volunteer Portal: https://volunteers.joomla.org/teams/documentation-team Or in the worst case try to reach somone else from the community and we are happy to direct you to the right people. ;)

For dev docs I would reccommend checking out the github repo for now when you need the access to the docs.joomla.org restored try to reach someone from above or me via the volunteerportal and I'm happy to pass the username to the people who can restore access :)

We are forced to use some extensions because there is no alternate, and the client doesn't want to pay for building something that already exists. Some of these extensions are CLOSED source, using ioncube loader and cannot be forked, which I personally dislike.

Oh that is going to be complicated, didnt know that ioncube is still a thing. :)

Our clients see a threatening message (for them) to update to Joomla 4, yet their admin (myself) tells them that they can't do it yet, because of some extensions. Do you know what they truly hear instead? Your site is getting old and I cannot fix it.

Go and hide / change the message. Its documented and intended to be changed to your like: https://magazine.joomla.org/all-issues/august-2021/joomla-3-10-end-of-support-handling

Yes, you are right. Brian is also right. It's not the volunteers' fault. I would just like to see Joomla handling this in a smarter way that penalizes the outdated extensions, not the end-users. That's why I opened this issue. I think this is important.

Please dont get me wrong I do fully see your point, but please let me give a question back to you. At what time when not within a major would we be allowed to remove that methods? Specigficly as they are now with J4 and the CLI and API clients not a very good idea to still use as that method does not respect that or is sensible implemented into the CLI and API app?

Anyway, I will try to make a PR that addresses this in a simple way: When a plugin breaks the back-end, identify the culprit plugin and offer a button to disable it, directly. Do you think this could work?

Sounds like a good idea which I'm happy to review, we should of course avoid expensive calls to the "regular site usage". Maybe its worth posting a short intro about to the desired technical solution implemention first so we can make sure it does not hurt the regular users :)

[mavrosxristoforos]

I understand and appreciate your response very much.

At what time when not within a major would we be allowed to remove that methods? Specigficly as they are now with J4 and the CLI and API clients not a very good idea to still use as that method does not respect that or is sensible implemented into the CLI and API app?

Still sensible in the AdministratorApplication and SiteApplication though, aren't they?

Anyway, I'm digging into identifying the culprit plugin while in Joomla\CMS\Exception\ExceptionHandler.php. It seems like a reasonable place to start. Of course, we only want to try that when an error has occurred in the administrator, and when caused by a system plugin, because that's usually the case that makes the back-end unusable. I'm using the $error->getFile() to split the path back to its directories and see if I can identify the plugin from its directory. Does it sound reasonable so far?

After that, how should I render the button? The ExceptionHandler currently uses renderer classes that include the template's error.php file. I believe this needs to be template independent. I could enqueueMessage and add a link directly in the message, but that seems somewhat unconventional to Joomla standards. Any ideas?

[brianteeman]

solve the first problem before looking at how to render it

[zero-24]

Still sensible in the AdministratorApplication and SiteApplication though, aren't they?

The problem comes when a plugin is triggered via API or CLI this is not implemented and will break, this is the reason for the new isClient :⁠-⁠D

[zero-24]

After that, how should I render the button? The ExceptionHandler currently uses renderer classes that include the template's error.php file. I believe this needs to be template independent. I could enqueueMessage and add a link directly in the message, but that seems somewhat unconventional to Joomla standards. Any ideas?

Thats something to be looked into later. In the worst case we do set it as user variable within the session and pass it along. For that we will fined a solution. catching and detecting would be the first step to look into.

[mavrosxristoforos]

Alright. I'll focus on the first part. Have you seen this happen in plugins of other folders other than system? Have you seen it happen when including libraries?

[brianteeman]

concentrate on one thing - solve that and then move onwards

[mavrosxristoforos]

I believe I've solved that. That's why I ask. It's easy to extract the error file directories and look if it's a descendant of plugin/system/ and the third directory in the path is always the plugin name. Am I missing something? Is this approach too naive?

[mavrosxristoforos]

So... I have been able to identify the culprit plugin as long as any of its directory contents appear in the error trace log. I've done this in the ExceptionHandler class, as a separate method, in which I add the culprit extension to the app's userState.

Here are two things I need your wisdom with:

1. What would be the most effective way to present this to the user? We still need to keep the existing error interface with the trace log for extension developers, so adding a button to disable this extension, for example next to the "Return to Dashboard" one, seems ideal. However, that is template dependent.
2. What if the user seeing this is a guest? If the administrator interface has collapsed, you can't even log in. Chances are this will happen while we still have a logged in admin, so perhaps we should only address that case, first?

What do you think?

[brianteeman]

This absolutely should not be presented to a non-authorised user This does not Maintain a usable Administrator interface on system plugin errors

[mavrosxristoforos]

WordPress addresses this by automatically sending an email to all administrators with a recovery link. That's what I had in mind when asking whether this option could be shown to a guest.

[brianteeman]

it is still not doing what you said it would "Maintain a usable Administrator interface on system plugin errors"

[mavrosxristoforos]

It presents a way to recover when such errors occur. It penalizes the extension producing the error, but still allows extension developers to see their trace log and debug. I thought from Tobias's comment that this would be enough.

Otherwise, we would have to try catch around the plugin file loading in PluginHelper, and also around the extension event triggering in CMSPlugin. However, doing so would disregard the existing exception handling structure. Would you prefer that approach?

[brianteeman]

It really is no different to what already exists.

The only reason this has been kept open is because you stated you could "Maintain a usable Administrator interface on system plugin errors". Anything less than that is unlikely to be accepted.

[mavrosxristoforos]

What already exists is an unusable interface with no recovery options. You have to either manually disable the extension in the database, or debug the problematic extensions with a file manager or FTP.

I could easily disable the plugin and redirect to the dashboard with a simple message stating what happened, and this WOULD maintain a usable administrator, but make the extension developers' life harder.

[brianteeman]

I could easily disable the plugin and redirect to the dashboard with a simple message stating what happened, and this WOULD maintain a usable administrator, but make the extension developers' life harder.

That is what you promised. Anything else is not an improvement

[crystalenka]

Adding a button for authorized users to disable the offending extension is not a bad idea.

I don't think that we should automatically disable any plugin on behalf of the user without explicit authorized action.

[crystalenka]

Brian, small steps are better than none at all.

[brianteeman]

@crystalenka you cannot have a button to disable a plugin after it has already broken the site

but as @mavrosxristoforos is insistent that it is extension developers like himslef that are keeping joomla alive I will let him work it out -

[mavrosxristoforos]

@crystalenka Thank you Crystal.

@brianteeman Then how do you have a button to return to the dashboard? Joomla is handling the error and presenting an interface, but misses a way to fix it.

As per your other comments, I didn't respond, but since you insist I have to explain. I said: "Joomla only still exists because people develop extensions for it". How is that not true? I didn't say that its success is in the extensions, nor that us, extension devs are keeping Joomla alive. Those are things that you said. I only wanted to stress out how important it is for Joomla to be easy and transparent in order to attract new and talented extension developers. Isn't that absolutely necessary for a thriving CMS? Isn't the large amount of Joomla extensions mentioned right there in Joomla.org home page, under "Why Joomla?" together with the volunteer devs? Is there any CMS out there that doesn't require extensions and nice templates?

Please bypass that, as I'm here trying to make a useful contribution, only to feel that I'm probably not welcome to do that either.

[brianteeman]

talk is cheap. still waiting to see your code that makes a usable admin interface on system plugin errors - as is everyone else.

[crystalenka]

@mavrosxristoforos your contributions are welcome. It's okay if you don't have all the answers on how it might work yet, I am happy to help you brainstorm. If we can do a button there that's great, if not we can figure something else out.

[alikon]

as you submitted a pr #39484 can you please close this issue thanks in advance

[mavrosxristoforos]

Sure, but @chmst , @zero-24 , @crystalenka please review the PR and make your suggestions. I believe this issue is very important, and the team seems to overlook how serious it is for the admin interface to become bricked.