Incorrect notifications in Update Checker

WoodyF4u commented 3 years ago

Steps to reproduce the issue

Open the Joomla Upfdate Checker for the Next Joomla release. See which extensions and templates are suitable for the migration to Joomla 4 and which need attention. In the list you will find extensions that are actually suitable for migration, but are incorrectly reported.

Expected result

Extensions such as Akeeba Backup, Akeeba AdminTools, BA Forms and RSSEO are now incorrectly reported as not suitable for updating These extensions should actually be at the bottom as no action is required.

Actual result

That is not going well at the moment.

System information (as much as possible)

Joomla 3.10.2 in combination with some extensions. Websiyte was once built in Joomla 2.5 and then continuously updated.

Additional comments

I was hoping this was fixed since the update to Joomla 3.10.2 See :https://github.com/joomla/joomla-cms/pull/35510

This is what they say at Akeeba: https://www.akeeba.com/support/akeeba-backup-3x/Ticket/35784:questions-about-the-migration-to-joomla-4.html

And here you can read that they are ready for Joomla 4: https://www.akeeba.com/news/1748-joomla-3-10-and-4-0-common-issues.html

This is what they say at RSJoomla: You can disregard that message since RSSeo! provides compatibility with Joomla! 4 for a while now:

https://www.rsjoomla.com/support/documentation/rsseo/changelog/rsseo-changelog.html

zero-24 commented 3 years ago

Can you share some more details like a screenshot of what you see and how can reproduce the issues?

From the akeeba issue my understandig is that the package_id and extension id mapping is not setup correctly? If so its hard to fix such issues within the core. I'm not sure whether extension developers would be happy when we would go and try to guess the relationships when not set by the extensions themself?

For the other extensions not affected by the missing package id setup please share with us the updateserver URLs you can find them under extensions -> manage -> update sites.

WoodyF4u commented 3 years ago

Please check this screenshot:. Screenshot-Joomla_Upfdate-Checker

Updatesite URLs: dmin Tools Professional https://cdn.akeeba.com/updates/pkgadmintoolspro.xml

Akeeba Backup Professional https://cdn.akeeba.com/updates/pkgakeebapro.xml

BaForms https://www.balbooa.com/updates/baforms/joomla_forms_update.xml

RSSeo! https://www.rsjoomla.com/updates/com_rsseo/Component/com_rsseo_2.5-3.0.xml

zero-24 commented 3 years ago

dmin Tools Professional https://cdn.akeeba.com/updates/pkgadmintoolspro.xml

hmm thats interesting as that should work fine and the version 6.1.1 is marked as compatible with 3.10 and 4.0. I dont have access to any pro versions of akeeba so I can not test and debug myself maybe someone else can take a look into that?

Akeeba Backup Professional https://cdn.akeeba.com/updates/pkgakeebapro.xml

This update server also looks ok and from your screenshot its not showing any issues about akeeba backup? I dont have access to any pro versions of akeeba but i can confirm the free version is working ok and is listed as "no update required".

BaForms https://www.balbooa.com/updates/baforms/joomla_forms_update.xml

This update server looks interesting as there is no download URL setup just a url to the login page? Beyond that BaForms from me is also listed under "No update required" which should be fine.

RSSeo! https://www.rsjoomla.com/updates/com_rsseo/Component/com_rsseo_2.5-3.0.xml

The RSSeo thing also seems to be an payed extension too which i dont have access to so i can not test that thing. It declares to be compatible with all versions of Joomla which is not a wise chooice in my option but should work nethertheless.

The more interesting part are the "Upgrade server issue" messages shown in the pre upgrade checker, would you be able to share a backup (removing the private data first) of the site to tobias.zulauf@community.joomla.org? So I could try to reproduce and debug the issues?

nikosdion commented 3 years ago

@zero-24 You said this

I'm not sure whether extension developers would be happy when we would go and try to guess the relationships when not set by the extensions themself?

I take personal offence because you are AGAIN blaming third party developers for a core Joomla issue, one I explicitly stated on June 28th when reporting — for the third time in a year! — that the pre–update checks in Joomla are misleading: https://github.com/joomla/joomla-cms/issues/34599#issuecomment-869491722

The Joomla projects EXPLICITLY forbids third party developers from using their own extensions updater and/or installer on pain of having their extensions permanently removed from the Joomla! Extensions Directory. As a result, we are FORCED to use the extensions updater and installer Joomla provides.

The Joomla extensions installer / updater is SUPPOSED to populate the package_id column of the #__extensions table for extensions included in a package. However, I have confirmed that it only does that when FIRST INSTALLING a package, not when updating a package. This means that are several use cases where Joomla will NOT populate the package_id correctly:

If you extract a package ZIP file and install each extension / some of these extensions directly THEN figure out you cocked up and install the package ZIP, same or different version of the package.
If you use Discover and then try to install the package ZIP, same or different version of the package.
When you have a site with a long history reaching back to the early 1.6/1.7/2.5 and 3.x days when most extensions were NOT provided as packages for the simple reason that package extensions were not supported in Joomla 1.5 which was still being widely used at the time and developers needed to support. When you eventually started using the package extension Joomla would NOT populate the package_id.
If you mess up and delete an extension table row e.g. by uninstalling a sub–extension instead of the package, then install the package ZIP file to fix what you messed up Joomla will NOT populate the package_id.

These are all issues I have seen on real world sites of my clients AND local development sites.

Here's a simple way to reproduce this.

Install the Admin Tools Core package.
Uninstall the System – Admin Tools plugin extension.
Extract the Admin Tools Core package and install just the plg_system_admintools extension. This is a reasonable course of action for a user, mind you.
Install the same Admin Tools Core package again.

Now examine the #__extensions table. You will see that the package_id for the plugin is 0.

Try updating your site, be it Joomla 3.10 or 4.0. You will be told that the System – Admin Tools plugin is incompatible with the new Joomla version.

This is what I reported 3 months ago! How the hell is this my problem and not Joomla's, for crying out loud?!

Joomla should fix the package_id when you install a package, be it a from scratch installation or an update. It's really that simple. It's not my responsibility as a developer to do that.

You have my email. If you need help reproducing the issue I can spend some time today to write step–by–step instructions in more detail and even send you a backup of a minimally viable Joomla 3.10 or 4.0 site (or both; I leave it up to you) where you can reproduce this issue.

I really want to see this problem fixed because it's really damn annoying and outright slanderous! As a 3PD all I can do is offer a tool to remove obsolete extensions. I cannot offer a tool to fix package_id associations without possibly running afoul of the JED's Terms of Service. If the Joomla project decides that it's a 3PD's responsibility to provide correct association in the core #__extensions table then I want that in writing, printed and signed by the entire Board of Directors. If you can't offer that all I can do is help you fix a blatant core bug and demand an apology for your continued slandering of my company and every other third party developer.

WoodyF4u commented 3 years ago

If it is still necessary I can send Tobias a backup of a website. But maybe Tobias and Nicholas will work it out together. Let me know if I need to offer a backup of a website.

nikosdion commented 3 years ago

@WoodyF4u Send Tobias a backup of your site because it's a prime example of what a real world site looks like. I think this would be a great resource for Tobias and other leadership members to see what us 3PDs and site integrators see every day. Every time I explained why the pre–update check is problematic I was ignored or my comment was marked as off–topic. Okay, then, let them see a real world site which demonstrates in no uncertain terms I was right all along and maybe it will get them moving to fix what I first told them a year ago needs fixing.

If Tobias wants me to, I can additionally provide the minimally viable clean site with real world messiness to make it easier to troubleshoot. I know why things get messy and I can reproduce it in a minimal site but it's not a real world site, it's a simulacrum. It's good for developing a solution, it's not good for demonstrating that the problem does happen in the real world.

WoodyF4u commented 3 years ago

I just sent a backup of the website to Tobias. Hopefully he can do something with that. Please let me know if additional information is needed.

zero-24 commented 3 years ago

Extensions such as Akeeba Backup, Akeeba AdminTools, BA Forms and RSSEO are now incorrectly reported as not suitable for updating

I can not confirm this inital report. For me (just after restoring your backup) Akeeba Backup, Akeeba AdminTools, BA Forms and RSSEO are all correctly listed under the "No Update Required" section.

I can also not confirm the Update server error and Pre Update checks faild messages that you get on your screenshot. I only get Update Information Unavailable and No Update Required

Could it be that there are some additional error messages on your site maybe in the browser console?

I have also checked some of the extensions listed as not compatible:

Akeeba Backup Notification Module

Does no longer exist on a filesystem level and seems to be no longer used by Akeeba Backup itself, so it might just be a leftover somehow?

System - Akeeba GeoIP provider plugin

Is still installed in Version 1.0.1 (that seems to be very outdated: "Released on: Monday, 30 December 2013 06:40"). Its update server points to this update xml: http://cdn.akeebabackup.com/updates/akgeoip.xml. This update xml does no longer exist and therefore can not declare any Joomla 4 compatibility which means this extension is correctly listed under "Update Information Unavailable"

System - Asynchronous Google Analytics

Does not provide any update server and therefore can not declare any Joomla 4 compatibility which means this extension is correctly listed under "Update Information Unavailable"

JSN Sun Framework

Does provide an update server which points to: https://www.joomlashine.com/versioning/extensions/sunfwframework.xml that update server declares only be compatible with 3.[0123456789]|10 and not with 4. So it does not declare any Joomla 4 compatibility which means this extension is correctly listed under "Update Information Unavailable"

BaForms (Files)

That seems to be a leftover again. Its an extension with a creation date of 2015 and without any update server and therefore can not declare any Joomla 4 compatibility which means this extension is correctly listed under "Update Information Unavailable" while the component seems to haven an update server which is up-to-date.

I'm not sure whether extension developers would be happy when we would go and try to guess the relationships when not set by the extensions themself?

I take personal offence because you are AGAIN blaming third party developers for a core Joomla issue

I have explicitly stated this about extensions that we would have to "[...] guess the relationships when not set by the extensions themself". I'm thinking here about components that install additional plugins but not using the package extension type and use component. I'm sure you would agree with me that the core should never go and manipulate anything thats under the domain of the extensions for example to try to guess relationships between extensions when said extension do not declare the included extensions in a package.

Here's a simple way to reproduce this.

Install the Admin Tools Core package. Uninstall the System – Admin Tools plugin extension. Extract the Admin Tools Core package and install just the plg_system_admintools extension. This is a reasonable course of >action for a user, mind you. Install the same Admin Tools Core package again. Now examine the #__extensions table. You will see that the package_id for the plugin is 0.

I can not fully confirm this on 3.10.2 following your steps. After uninstalling plg_system_admintools and reinstalling it manually yes the package ID is not set which would be correct as its not installed as a package yet. But once I install the full admintools package again the package ID is correctly restored even for plg_system_admintools?

(Thats after unintalling and reinstalling the standalone version of the plugin)

(Thats after installing the core package over all of that)

Wouldnt be the correct way be to deny the uninstall of individual extensions attached to a package in the first place so we are not going to get into that "mixed state"?

Or do I miss something here? You said you can provide a backup where that issue can be reproduced would you able to provide that backup?

one I explicitly stated on June 28th when reporting — for the third time in a year! — that the pre–update checks in Joomla are misleading: #34599 (comment)

I must have admit that I have interpreted the issue reported in the inital report (#34599) to be much more of the wording rather than an issue with package updates. That issue seem to had many issues tracked in one issue alone and resulted into different patches done for the pre-upgrade checker. That issue you have reported later in response to another developer seems to be missed by me.

zero-24 commented 3 years ago

@WoodyF4u

Ok I have just saw that you also send me the account data for the live site too. There i can reproduce the issue with the HTTP 500 messages reported in the browser console:

When looking at the reported response i get:

But when calling it directly in the browser i get the correct JSON:

So could it be that there is something on the hoster site that is limiting ajax requests? As the 1:1 copy of your side on my localhost does not have any such issues and even calling it directly from the browser come back with the correct JSON?

IIRC also just the word "Error" is not an issue reported by the core cms, so my first guess would be that something on the hosting site is hitting you here can you please check?

nikosdion commented 3 years ago

I will get back to you on the package_id because that's something I had reproduced again on two dev sites with the final RCs of 3.10 and 4.0, hence the article on my site. I have some theories, I need to figure out how to test them on the laptop.

I can, however, tell you exactly what the 500 errors are. They are caused by the server and this is a VERY common server setup.

Most shared hosting limits the request rate with various methods. In short, if you hit the same URL ‘too many’ times within a period of time it will return a 500 error. In some cases it might auto–block your IP address. This is something I had seen since I was developing JoomlaPack since 2007 and I can assure you it is STILL an issue in 2021 — that's why there is a minimum execution time in Joomla Update's restore.php or my newly contributed extract.php. Please read the code comments in #35388 to understand what I am talking about.

The problem is that Joomla Update in the pre–update check performs a barrage of concurrent requests. This works on localhost where this feature was evidently only ever tested but not on most shared hosting where Joomla users will be using this but apparently this feature was NOT tested on 😛

Here are some alternative fixes for it:

Run the requests serially, making sure the minimum time between requests is 2 seconds. This is enough for MOST, but not all, shared hosts. Some shared hosts require a rate as low as one request every 7 (yes, seven!!!) seconds to prevent this kind of issues. The obvious problem is that the pre–update check would take several minutes to complete.
Retrieve information in batches from the server, using the same min/max/bias execution time I am using in restore.php, extract.php and of course my own software. Joomla would be hitting a single endpoint which returns the information for one OR MORE extensions. On the server side, Joomla would start a timer. Try to get update information for one extension. Check the timer; if it's below 5 seconds repeat the process. Otherwise stop for a number fo whole seconds equal to 7 minutes the elapsed time; if that number is negative return immediately. The client–side would hit this URL with the ID of the last extension returned from the previous step until the server result is empty, signifying that all extensions are checked. The downside is that you have a small delay for the initial batch of information BUT this is far more resilient in the real world.

Whenever you need to run async requests please ask me to take a look. I've been doing that for a living for 11 years and have been working on it since 2006. I was one of the first developers to use asynchronous (AJAX) requests with Joomla for long–lasting processes. I know how things break in the real world and how to work around them. I have a unique sort of experience on this subject. I am willing to help if you let me 😉

WoodyF4u commented 3 years ago

Nicholas, thanks for your explanation. Tobias, I think I'll hold off on reporting to my hosting company for a while. If it is indeed a problem that occurs by default on shared hosting, then it makes no sense to report it as an issue. By the way, I tested it in different browsers and I don't see any difference in output.

I hope that you and Nicholas can build in the right delay so that it will work well on both local installations and shared hosting. Apparently the Update Checker

zero-24 commented 3 years ago

Whenever you need to run async requests please ask me to take a look. I've been doing that for a living for 11 years and have been working on it since 2006. I was one of the first developers to use asynchronous (AJAX) requests with Joomla for long–lasting processes. I know how things break in the real world and how to work around them. I have a unique sort of experience on this subject. I am willing to help if you let me 😉

Feel free to take a look for that ajax stuff. I was not aware of such limitation its also not an issue on my 'live' hosting so its even harder for me to reproduce and debug.

For the longer checking stuff can we somehow detect such hostings, like running a few checks and when there are issues call them again with the delay? So we only do the "wait 7 seconds for each requests" on such hosts or for the failed requests?

The other thing would be, as you said, one endpoint that returns everything I'm not sure whether we run into different issues with larger sets of extensions or actually time outing updateservers?

I will get back to you on the package_id because that's something I had reproduced again on two dev sites with the final RCs of 3.10 and 4.0, hence the article on my site. I have some theories, I need to figure out how to test them on the laptop.

Thanks when you could get it reproduced please create a dedicated issue for it on how we can reproduce this so it does not get lost between other issues again.

zero-24 commented 3 years ago

Nicholas, thanks for your explanation. Tobias, I think I'll hold off on reporting to my hosting company for a while. If it is indeed a problem that occurs by default on shared hosting, then it makes no sense to report it as an issue.

I can not confirm the 'on all shared hosting' bit as my shared hosting is not affected and you are the first who i see reports that issue. But Nicholas seems to be aware of the issue on some shared hosting and also has a solution which would be great to be implemented for such hosting setups.

I personally would report it to my hosting but thats up to you when there is a PR incomming on the changes for such hosts we would need a hosting setup that we can test it on too :-)

By the way, I tested it in different browsers and I don't see any difference in output.

Yes its a server but not a browser issue that, with the proposal from Nicholas, we can try to workaround on the application level.

PhilETaylor commented 3 years ago

but not a browser issue

Browsers tend to limit active pending ajax requests to the same domain at 4-6...

zero-24 commented 3 years ago

but not a browser issue

Browsers tend to limit active pending ajax requests to the same domain at 4-6...

Ok so it could hit us on a "slow" server where the requests take longer to complete the check for the single extension.

nikosdion commented 3 years ago

On the hosts where the longer delay is required you typically get your IP automatically banned by sending multiple requests. So that wouldn't work either. You would just cause the client to be locked out of their site.

Regarding the second solution, I can guarantee it's not going to be a problem. This is the approach I have used in my backup software, my archive extraction software, my file scanner software and Joomla Update itself. It's battle–tested over the past 15 years on hundreds of thousands of live servers, hundreds of millions (if not already billions) of times over.

Here's a bird's eye view of how that works server–side:

Is the last extension checked non–zero?
- Fetch the extension IDs from the session variable. If it's empty fall back to “this is zero” branch below.
- If the extension ID does not exist fall back to “this is zero” branch below.
- Get the extension IDs array slice AFTER the last extension ID in the request.
- If the array slice is empty return an empty result immediately.
- Update the session variable with the new array slice.
Is the last extension checked zero?
- Get a list of all extensions to check using the same code we use to create the HTML front-end.
- Find the extensions without update sites and push them to the output array.
- Store the rest of the extension IDs to a session variable (e.g. com_joomlaupdate.preupdate_ids)
Get the list of extensions from the session variable.
Start a timer
Shift an extension ID from the array
If the array was empty break the loop
Fetch the update information for this extension, parse it and push the result to the return array
Do I have enough time? If yes, go to step 5.
Have I already spent the minimum amount of time? If not, sleep for the remainder time.
Return the array with extension results.

FWIW writing out detailed instructions like that is how I develop features. These comments go as // TODO comments in a blank method, I write the code bellow each one and then refactor the method out to smaller methods and do overall code polishing, testing and so on.

Regarding the some / all shared servers: it only happens to SOME shared hosting servers. They are a minority (thankfully!). I do not have any sites under my control on such a server BUT I have 15 years of experience working around them so I just know what to do and why. I can also reproduce the effects using mod_bw (available on Ubuntu Server, or at least it was on 16.04 when I last used it). I found this to be a good approximation when doing development but I haven't had the need to use it the past four years so I don't know if it's still available on 20.04.

Regarding the pipelining of XHR requests: yes, browsers will only make 4–8 concurrent requests even if you try to send 100 of them at once. The result from the server's point of view is actually the same: there is a barrage of requests for the same script (administrator/index.php) coming from the same IP in rapid succession. This is what triggers the server protection.

By the way, the method I propose is actually going to be faster overall on a slow server, even one not affected by the rate limiting issues, on under high latency / low bandwidth network conditions. I know it's comically counter–intuitive but on these servers the overhead of each request is significantly higher than the work performed for the bulk of extensions. Batching them up makes the overhead smaller than the work done which is overall faster.

PhilETaylor commented 3 years ago

Also - DO NOT TRUST that Joomla Extension Developers Update Sites are even working. I mean, like Nic, I have CONSIDERABLE experience with update site urls... some will quite happily sit there for 60 seconds wasting your time before telling you they are a 404/503/301/dead...

Not all update site urls will even be valid anymore.... at a hunch I think the last time I checked the 944,102 unique update site urls that I have access to, over 50% were not a 200 OK response with a useable XML file.

nikosdion commented 3 years ago

Sorrrrrry forgot to mention one VERY MAJOR THING (I'm replying to too many Joomla stuff today to keep my thoughts in order): In my approach I would limit the time to retrieve update information within the pre–update checks context AND ONLY THERE to 5 or 10 seconds — I need to pore over some notes I've taken on past live site troubleshooting to determine that. For the update sites that timeout I'd put the extensions under “no update information available” AND show a previously hidden link to the extension updates page with the note that some updates could not be retrieved “in a reasonable amount of time” so please go there to retry retrieving them.

In case it's not obvious, the extensions whose updates are already retrieved within the cache period set up in the options for extensions updates would NOT be retrieved again from the network. What's the point in caching the updates if we're gonna hit the developers' update servers like Neanderthals every time Joomla Update detects an update? How do we know? Well, there's the last update timestamp in the database and Joomla already takes it into account, for one thing 😉

nikosdion commented 3 years ago

Another aside for @PhilETaylor

Joomla does not replace update sites when you install a new extension version with a different update site. It just adds another update site. I know for a fact that some of the broken update sites are old ones from previous versions of my extensions. Some extensions were using GitHub URLs to provide update sites. Earlier this year I moved them to my CDN. The old URLs throw 404. On my premium products I have code to explicitly find and remove the old update sites.

I think that's one reason many people pressed that Rebuild Update Sites button, frustrated at the constant errors. That was broken for a while, not it's fixed.

These are all issues that are too real for us 3PDs but are impossible to know if you are not a 3PD and don't deal daily with real world sites. Clean room development is great for reproducibility and having different volunteers test a feature or patch but only shows you so much of what can possibly happen on a site in the real, very messy world...

richard67 commented 3 years ago

I think that's one reason many people pressed that Rebuild Update Sites button, frustrated at the constant errors. That was broken for a while, not it's fixed.

@nikosdion "not it's fixed", or "now it's fixed"?

zero-24 commented 3 years ago

On the hosts where the longer delay is required you typically get your IP automatically banned by sending multiple requests. So that wouldn't work either. You would just cause the client to be locked out of their site.

Ups yes thats not intended ;)

Regarding the second solution, I can guarantee it's not going to be a problem. This is the approach I have used in my backup software, my archive extraction software, my file scanner software and Joomla Update itself. It's battle–tested over the past 15 years on hundreds of thousands of live servers, hundreds of millions (if not already billions) of times over.

Ok fine for me, you have much more expirience on such sites so feel free to go with that solution :)

Also - DO NOT TRUST that Joomla Extension Developers Update Sites are even working. I mean, like Nic, I have CONSIDERABLE experience with update site urls... some will quite happily sit there for 60 seconds wasting your time before telling you they are a 404/503/301/dead...

Yes thats why I'm not sure whether merging everything into one call would suffer from such issues.

In case it's not obvious, the extensions whose updates are already retrieved within the cache period set up in the options for extensions updates would NOT be retrieved again from the network. What's the point in caching the updates if we're gonna hit the developers' update servers like Neanderthals every time Joomla Update detects an update? How do we know? Well, there's the last update timestamp in the database and Joomla already takes it into account, for one thing 😉

Well its not that easy. We are checking in the 3.10 pre-upgrade checker against 4.x where the "normal" update only checks against 3.10. So everything in the cache should only be an update which would be 3.10 compatible. Sure when you are running that check again and again feel free to take the cache but that should IIRC been taken care of the updater lib already.

For the update sites that timeout I'd put the extensions under “no update information available” AND show a previously hidden link to the extension updates page with the note that some updates could not be retrieved “in a reasonable amount of time” so please go there to retry retrieving them.

Fine for me. But what previously hidden link to the extension updates page you are looking for? Do you mean the update server overview where such update servers should be marked as "disabled"?

nikosdion commented 3 years ago

@richard67 I meant NOW it's fixed. Sorry, typing blindly in English while also speaking Greek to my mother watching our daughter. My brain misfired there.

Well its not that easy. We are checking in the 3.10 pre-upgrade checker against 4.x where the "normal" update only checks against 3.10. So everything in the cache should only be an update which would be 3.10 compatible. Sure when you are running that check again and again feel free to take the cache but that should IIRC been taken care of the updater lib already.

Yeah, that's sort of the point. I need to take a look at how this works under the hood. I will possibly need to make a schema change if I recall correctly what is stored and how. I will need to carve up some time to actually look into the code, don't hold my preliminary musings about how it would work against me LOL!

Fine for me. But what previously hidden link to the extension updates page you are looking for? Do you mean the update server overview where such update servers should be marked as "disabled"?

Right, I failed to convey this correctly. I would have a DIV hidden by default with this content and the link to the extensions update page. If the server reply indicates an extension update can't be fetched at all (timed out or got an invalid / non-XML reply) I'd show this previously hidden DIV. That's a pattern I use a lot with my software for warnings generated by async client–side code which I've filed on my head as “previously hidden link” because I first used it to show links to troubleshooting pages when something went wrong. Of course you wouldn't know this since you're not in my head 🤣

nikosdion commented 3 years ago

@WoodyF4u If you have problems with “orphaned” extensions try installing their packages again (or, if updates are available, install the updates). The association between extensions and the package are restored. You may have to clear Joomla's cache for that to take effect though because of the way Joomla is caching the model items.

@zero-24 I can definitely make a PR to improve the pre–update check as I described. Should it be against Joomla 3 or 4? I can do one and then you or another contributor could copy it to the other. It may take me a while (read: a week or so) because there's a backlog of Joomla issues I am trying to help with on top of running a business, rewriting a few remaining extensions as native Joomla 4, preparing our own site for Joomla 4 and our daughter just having started pre–school which requires me to do a hard stop in the evening to pick her up and spend some time with her as I no longer have her throughout the day. It's a transitional phase in many respects and I'm trying to juggle everything the best I can :D

zero-24 commented 3 years ago

A PR against 3.10-dev would be the way to go for now as for now thats the place where we need this most. When merging it up to J4 this has to be adapted for sure.

And no need to rush a PR as you know the next rc release is not expected earlier than the 16. October but if not ready by than its fine to go into the following release too.

nikosdion commented 3 years ago

A quick note that I have not forgotten or lost interest in this. Our daughter got sick, again (the joys of first year in preschool), I'm running like a headless chicken. October 16th is a borderline realistic target and I'll do my very best to meet it.

Edit: spelling errors. Sorrrrrry!

zero-24 commented 3 years ago

@nikosdion I'm sure you know but let my say it too your family is much more important, please focus on that first and come back to this once your daughter is good again. When its not ready by the 16th it can go in the following releases without any problem, specifically by "ready by 16 Oct" means ready to be merged by that date as the 16th is the date of the RC.

WoodyF4u commented 2 years ago

Is there any progress on this question yet?

nikosdion commented 2 years ago

@WoodyF4u My plan is to get started on it in December.

I have not managed to work on that because of how complicated a small sniffle, cough and low fever — typical things during the first year of preschool — are in a pandemic. A low fever is no longer 1–2 days out of school with some help from the grandparents if needed. It's a full week out of school. Now imagine what happens when this is followed by one more week out of school because of a confirmed Covid-19 case in the class across the hall from my daughter's. Yup. She's not been in school since November 16th and she won't be back before November 29th — assuming we can get a doctor's appointment, for the earliest one we have found so far is December 1st.

First year of preschool during a pandemic: 0/10, would not recommend.

TheStingPilot commented 2 years ago

Hello all,

I have the same issue: Update Server error under Pre-Update Checks Failed. However, I make a backup from my site with Akeeba and restore the backup on a CentOS 8 machine that is running on a HyperV VM, all the checks are fine and the Update Server error does not come up.

Tomorrow, I will continue with the upgrade and simply ignore these 'errors'.

With kind regards, Willem-Jan

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/35572.}

nikosdion commented 2 years ago

@TheStingPilot Thank you for confirming my findings! What happens to you is that the first server you try declines the barrage of requests, throwing errors and causing Joomla to report misleading errors with extensions. The CentOS 8 machine I suppose runs the default CentOS LAMP stack which doesn't rate–limit or reject requests. Therefore it runs correctly. That's why I said I need to rewrite all that logic to do it in sequential batches instead of trying to launch several dozen XHR's all at the same time, making the server think it's being DoS'ed.

Hopefully I can find the time next week. We are entering the time of the year where support requests go from massive peak to a valley (thank God for the holiday season...) and I'll finally stop being swamped :)

joomla / joomla-cms