Ways to improve backend performance

[ghost]

I'm finding October CMS runs slow in the backend when loading between web pages and would like to open a discussion about this.

Having a general think about things, we have many files that are not minified (for testing reasons).

The files I find that are slowing things down are the main javascript files in the backend things such as: storm-min.js etc.

p.s. storm-min.js, october-min.js and october.flyout.js seem to be the worst performing files by quite a margin over the rest.

october.flyout.js slow issue is related to an open issue found here: https://github.com/octobercms/october/issues/4298 and https://github.com/octobercms/october/issues/4052

Other factors that make October feel slow is that I'm using CloudFlare which I have seen a few times: Railgun Errors.

Another factor maybe the large number of plugins I have running.

Speed Loading Guide

Below is a table showing the speed performance guide by Google on how fast a web page should load, note the backend web pages should in theory loaded under 100mS.

As developers we aim for 'instant loading'.

p.s. My front-end loads instantly, yet the backend is slow and sluggish.

Service Worker

One solution I would like to raise and discuss is to make some of these files offline and cached.

By using a Service Worker in the Backend.

I know this would cause an issue when October CMS updates and to get around that issue I would suggest adding a Service Worker version number the same as the current October CMS. By making this variable dynamic and auto updating when October CMS is updated then the Service Worker would clear the cache when October CMS is updated - thus giving those files a refresh.

e.g.

storm-min.js?v=456

would be:

workbox.core.setCacheNameDetails({
    suffix: '<?= $script . \'?v=\' . $coreBuild; ?>',
});

I would suggest making the main JavaScript and CSS files cached and offline.

"304 Not Modified" response

Another idea, would be to using a caching method to speed up the loading of the same web page give a 304 Service Code instead of another 200 Service Code.

Final thoughts

These things should speed up October CMS loading times and not rely so heavily on the Network Connection (not giving any issues).

I leave it open to people's thoughts and ideas.

[ghost]

p.s. Please commit below is anyone sees any performance issues with the backend, would be interested to know how many people are affected by this related issue?

[alxy]

When you are talking about loading times/performance/speed - are you talking about the time it takes for the server to provide the requested files? (Which will depend on their respective sizes and your internet connection.) The specific asset files you mention are typically cached in the browser, so they should only significantly affect performance on the first load. And this is what happens for my october installations as well: basically all backen assets are loaded from disk, which is pretty fast.

[LukeTowers]

@ayumihamsaki I would say the best place to start is by profiling the backend to see where the biggest performance slowdowns are and work on minimizing them one by one with specific PRs.

[FlusherDock1]

I don't have any performance issues in the local or production environment. How many plugins do you have? How many of them have their additional assets?

The idea of ​​using ServiceWorker is very good. I think it is possible to implement it also to the frontend core framework.

[FlusherDock1]

Lastly, I'm using Google Canary, so that may also be causing some issues!

I just tested some of my biggest projects, with a lot of records and plugins on Chrome Stable, and I always have around 50ms on scripting task.

Can you please test it on other browsers as well?

[LukeTowers]

@ayumihamsaki it's highly unlikely that October is going to be rewritten in any major way, especially for the sake of checking of boxes of random online checklists / design patterns.

We are more than happy to deal with individual performance issues as they come up and are discussed in detail but I'm not seeing anyone else having the same problems as you. If you can make PRs for individual changes that you think will improve the performance along with a detailed background of the change and performance profiling for before and after (with server specs, browser details, and active load) then go for it.

I know there are definitely some individual pieces of the october backend where performance could be improved (for instance one of my projects heavily utilizes the dashboard with lots of custom dashboard widgets and the performance of it loading is abysmal), but everything should be addressed on an individual basis with detailed profiling of the performance before / after the change to be sure we're making changes that actually improve October and not just changes for changes sake.

[Samuell1]

First things that should be optimalized is to recompile all assets and minify them trough something better for example "framework.extras-min.css" can save a lot in size.

(first is custom recompiled trough postcss)

[bennothommo]

@ayumihamsaki Regarding ""304 Not Modified" response - that's something that is usually controlled at the server level. The only way we can enforce that is by routing asset calls through the PHP pipeline, which would result in even worse loading times initially. We could possibly add a doc page suggesting best practices for hosting, of which this could be one of them.

[ghost]

@bennothommo I wrote my own blog plugin for October, due to two reasons why I didn't like RainLab's Blog plugin and RadiantWeb's ProBlog Plugin. The two issues I found was:

1. They both have duplicate content issues creating web pages in their folder structures.
2. They both didn't have 304 Redirects.

In my Blog Plugin I did use php as you suggested to create the 304 redirect, the code example I used was a modified version found here: https://www.php.net/manual/en/function.header.php

    // Getting headers sent by the client.
    $headers = apache_request_headers();

    // Checking if the client is validating his cache and if it is current.
    if (isset($headers['If-Modified-Since']) && (strtotime($headers['If-Modified-Since']) == filemtime($fn))) 
    {
        // Client's cache IS current, so we just respond '304 Not Modified'.
        header('Last-Modified: '.gmdate('D, d M Y H:i:s', filemtime($fn)).' GMT', true, 304);
    } else {
        // Image not cached or cache outdated, we respond '200 OK' and output the image.
        header('Last-Modified: '.gmdate('D, d M Y H:i:s', filemtime($fn)).' GMT', true, 200);
        header('Content-Length: '.filesize($fn));
        header('Content-Type: image/png');
        print file_get_contents($fn);
    }

I modified the above code into a more general use case. I have not seen any performance issues running this code and it only works when the Server gives a 200 Code for the same web page.

It's a small piece of code only a few lines, so maybe worth considering?

---

@Samuell1 I know over the years a dozen people have asked for this feature.

Did a search and could only find this related issue: https://github.com/octobercms/october/issues/2014

I opened up a Github Issue for that feature here: https://github.com/octobercms/october/issues/4383

---

@LukeTowers I totally agree with you, when I wrote down some of these ideas, I feel they are not low hanging fruit and would require quite a lot of coding. As you labelled this as a 'discussion' I thought I'd add them here anyway.

Totally with you on creating smaller issues and pr's from this discussion.

Also the dashboard loading is terrible. I have quite a few plugins for the dashboard area but two I want to talk about are:

• RainLab's Google Analytics Plugin
• Scott Bedard's Google Analytics Extension Plugin

I spoke to Daftpunk a few years ago about this. Just to be clear the issue is not from his RainLab plugin. It's from Scott Bedard's plugin. The RainLab plugin is required to make Scott's plugin work.

The issue there is basically it takes between 5 - 10 seconds for the dashboard to load all the data from Google Analytics.

The Plugin stops the dashboard from loading straight away and that maybe a fix we could do.

Making the backend load straight away and not get slowed down by third-party plugins.

---

Any issues that are good people can comment and give +1 and then can open another issue for them.

If can keep this issue open for a while then more people maybe can add their input and I be up for helping improve performance issues with October.

p.s. I got a dozen PR's open right now and sitting under testing. So I'm just waiting for them to get processed before starting new things. But always happy to help October Team.

[LukeTowers]

I feel like dashboard performance could be improved by implementing caching of the widgets contents with perhaps an in widget refresh button to clear the cache for that specific widget instance.

[ghost]

I was thinking loading the dashboard core widgets first and then loading third party widgets afterwards. First by loading the widget code and then filling the content in. Also preloading the core code minus the content.

I draw what I mean maybe easier to understand me:

[LukeTowers]

I'm not convinced that's the best way of doing things, seems needlessly complex

[ghost]

Fair enough, my idea is based on 'trust models' we use in coding nowadays. We assume the core code is clean and the majority of errors would lie in the third party plugins that would cause issues to the loading.

[ghost]

I've just seen some small issues with the code, where we could probably fix:

The backend signin screen has two files:

modules/backend/assets/js/auth/auth.js

with the code:

$(document).ready(function(){
    $(document.body).removeClass('preload')

    $('form input[type=text], form input[type=password]').first().focus()
})

and the file

modules/backend/assets/js/auth/uninstall-sw.js

with the code:

// Only run on HTTPS connections
if (location.protocol === 'https:') {
    // Unregister all service workers before signing in to prevent cache issues
    navigator.serviceWorker.getRegistrations().then(
        function(registrations) {
            for (let registration of registrations) {  
                registration.unregister();
            }
    });
}

These are really small pieces of code and instead of wasting two look-ups you could just inline this javascript in the head section to speed up things!

Another issue I have just seen is that the uninstall service worker script is only running on the signup webpage and not throughtout the backend. Which is letting my Service Worker from the front-end run in the backend. Which is caching files in the backend, but more worrying I could inject code into the Service Worker from the front-end to run in the backend.

So I would like to also block off the Service Worker from the front-end running in the backend as a possible security threat.

Opened Issue: https://github.com/octobercms/october/issues/4384

p.s. Will try to do a PR for this today. PR Done found here: https://github.com/octobercms/october/pull/4385

(I did quickly as a possible security threat to cms)

[bennothommo]

@ayumihamsaki

In my Blog Plugin I did use php as you suggested to create the 304 redirect, the code example I used was a modified version found here: https://www.php.net/manual/en/function.header.php

Nice! Yeah, we can probably do 304 Not Modified calls for pages themselves (ie. CMS, Static Pages, Blogs etc.) if they haven't been implemented already, as they have to run through the PHP pipeline anyway. My comment was referring to other assets like CSS and JS.

[ghost]

@bennothommo Cool, I get what you are saying now! Sorry I misunderstood the first time around what you meant.

Yes please can we have 304 on the pages and not on the css and js.

[ghost]

Going to show you a screenshot of the backend Dashboard, it shows the dashboard having a 8 second (wait) delay in Google Chrome while the assets are loaded via JSON:

Would it be a good idea to preload these assets when a user has opened the signin screen. Because the next screen they will see will always be the dashboard, after they have signed in.

I'm basing my idea on the new "Portals" method of creating websites. But not using iFrames - only pre-fetch instead.

[ghost]

Q: Can we do something about storm-min.js there so much garage code inside.

1. It's 188Kb in size, could we minify it please to under 100Kb.

2. We got encrypted code:

3. We got a huge amount of strange Hexadecimal Code:

4. Not sure what's going on with this, if it's commented out or not etc.

5. There are somethings ref webpack and we are not using it.

I can see why it's takes so long to load and Google Canary scanning the code for any nasty things.

When I try to load a backend web page I really dont need 2/3 of this junk. For example, do I really need to load every single language pack at the same time?

[bennothommo]

@ayumihamsaki storm-min.js is the compilation of several libraries into one - see here for the list. I can say that (3) is likely moment.js and (4) are templates for certain widgets.

I'm sure there are ways to abstract certain libraries out of there and to load them only when needed, but truth be told, ~180kb of JS file for a whole heap of backend functionality, I personally feel, is not that big a deal.

[ghost]

Also would it be a good idea for October to start using some code size limits and create some standards for everything, for example something like this:

[ghost]

@bennothommo Thanks for the link. Can't we split storm.min.js into two separate files.

1. Load the essential files in the <head>.
2. Load the extra files in an async near the </body> closing tag.

[LukeTowers]

@ayumihamsaki you're welcome to try to split it and see how the performance is affected. Performance improvements on hard data only please 😉

[ghost]

@LukeTowers Thanks I will try to do, my theory is like this:

p.s.

I'm sure there are ways to abstract certain libraries out of there and to load them only when needed, but truth be told, ~180kb of JS file for a whole heap of backend functionality, I personally feel, is not that big a deal.

That would be fine if we were all running HTTP 1.1 but with HTTP/2 it makes no sense to try and pipe down a massive file.

On a timing diagram it would look like this:

Hopefully I can prove it with some results.

[bennothommo]

@ayumihamsaki I agree with you, but from a pragmatic standpoint, we cannot assume everyone running October is running a HTTP/2 compatible server.

[ghost]

@bennothommo totally agree with you. I'm going to preload both files so hopefully that will also help things.

[ghost]

Just had an idea! 💡

We have all talked about to minify or not to minify the backend assets many times.

Why don't we just create both sets and then attach them to the config/environment.php

See this code line here:

https://github.com/octobercms/october/blob/7590dfd30fb08242c6c8e111115659d38ac4aeb8/config/environment.php#L16

Development Example:

'default' => 'development',

Assets are set to un-minify.

Production Example:

'default' => 'production',

Assets are set to minify.

Then all I need to do is create a wrapper around the storm andoctober js and css files.

Or we can create another config variable.

Would ask Luke to decide where to create a new variable in the config area, as I can't decide between: config/cms.php or config/environment.php or config/testing.php

Also the CMS could check the deployment of where it is located. I notice a lot of reviewers are using localhost to test things, so maybe do the following:

Localhost

Load the following assets:

• october.css
• storm.css
• october.js
• storm.js

This saves time for the testers to have to set an option in the config settings area.

Config (Development) - Hosted on a Live Server

Load the following assets:

• october.css
• storm.css
• october.js
• storm.js

Config (Production) - Hosted on a Live Server

Load the following assets:

• october.min.css
• storm.min.css
• october.min.js
• storm.min.js

Hopefully this then allows some performance enhancements to people not writing and testing pull requests for October Source Code.

Default setting would be: Config (Production)

[LukeTowers]

If we went that way @ayumihamsaki then we'd use app.debug to determine to load minified or unminified. Again, hard numbers only 😉

[ghost]

ok 😄

[Samuell1]

I findout that Yaml parsing is not cached and it takes a long time. It should make backend a little bit faster. https://github.com/octobercms/october/pull/4526

[ghost]

Lazyload top navigation icons and remove external loading of default avatar https://github.com/octobercms/october/pull/4562

[github-actions[bot]]

This issue will be closed and archived in 3 days, as there has been no activity in the last 30 days. If this issue is still relevant or you would like to see action on it, please respond and we will get the ball rolling.

[LukeTowers]

@ayumihamasaki2019