Misc. improvements in HTTP connections and caching

[mbabker]

Summary:

• Adds a new method to the base model to get a configured JHttp client
• Refactor caching mechanism to store data in a com_apps group versus "global" ungrouped and to only store the response body data versus the full JHttpResponse object
• Introduces fallback mechanism to attempt to make a HTTP query if a cache exception is thrown (requires the 3.7 release due to using the new JCacheException interface, not a blocker for deploying while the system is on 3.6 because of how PHP handles Exception catch statements)
• Adds error detection in the HTTP request and throws an Exception if the JHttp client can't be retrieved, the response code isn't 200, or there is an error running json_decode() (which would be indicative of a bad response)
• Adds support for error logging (nothing is configured to log for the com_apps category though so it won't be written anywhere)
• Changes JED URLs to use HTTPS scheme
• Minor code style tweaks in modified methods to use project code style standards

In theory this should run against 3.6.5, nothing is dependent on 3.7 except for the noted catch block but as I said that isn't blocking for deploying now. With these changes, the cache storage should be improved a bit which hopefully makes the underlying system a bit more stable.

Simplest way to test:

• Clone/Download the PR branch and drop the files in your local CMS install
• Discover install com_apps
• Install the Install from Web plugin
• Change public $appsBaseUrl in the plugin to point to your local CMS install
• Browse install from web

[beat]

I have uploaded to staging server this branch for those who can help test:

To switch to staging: Open your file plugins/installer/webinstaller/webinstaller.php

and edit line 21: public $appsBaseUrl = 'https://appscdn.joomla.org/webapps/';

and change the webapps to stagingto test it.

[aleksanderkuczek]

So I only need to do what @beat suggested in order to test it? (And I can omit all the steps described by @mbabker?)

[mbabker]

Yep, Beat's suggestion is much easier.

[aleksanderkuczek]

Okay. So after 10 minutes of clicking all the main paths seem to be working. Sometimes slowly, but still working. There are some other issues like the search results sorting (which is different than on JED), but I understand that it was outside the scope. All in all, I think it's good enough for now. Thanks @mbabker for fixing that in no time (and @beat for making testing waaaaay easier)

[RickR2H]

Clone/Download the PR branch and drop the files in your local CMS install: DONE! Discover install com_apps: DONE! Install the Install from Web plugin: DONE! Change public $appsBaseUrl in the plugin to point to your local CMS install: This URL was already present in the plugin (www\joomla-cms-staging\plugins\installer\webinstaller) so nothing to change Browse install from web: Browsed the Install from web for 5 minutes

@beat I got no strange behaviours at this point. Is this enough to test and say it was successful?

[beat]

@RickR2H Thanks a lot for your help here too, confirmation that all works fine is good :smile:

Are the behaviors also better than on the normal live server ?

(e.g. it seems that for some people on live server sometimes doesn't display some categories)

[mbabker]

When I looked on the live server last week I saw a lot of 0 byte cache files. So that's probably playing into the empty category problem. With any luck by this PR making the cache data a bit smaller that issue happens less frequently.

[RickR2H]

@beat What do you mean with "the normal live server"? Loading time of the webinstaller is now 4 seconds...

[Webdongle]

Made the change to the file Clicked a few categories Installed a few extensions All appeared no problems

One plugin would not install but it would not install without the file modified either.

[beat]

@Webdongle Awesome! Thanks for the feedback!

@RickR2H The normal live server is with the unmodified webinstaller.php (with webapps folder). Live server might appear to be quite faster as with high usage its cache is more "ready" to serve.

[Webdongle]

Difficult to tell time with xampp on windows 10 it's always slow. But what I did notice is there was a significant difference loading categories that only had a few entries ... they loaded much faster than the categories that had loads of entries.