Decompose JQueryUI to speed up pages

DrewML commented 5 years ago

Preconditions (*)

Any version of Magento 2

Steps to reproduce (*)

Not a bug

Expected result (*)

I expect us to only load JavaScript that is used on a particular page.

Actual result (*)

We load several hundred kilobytes of dead code

Further Details

jQuery UI is one of the biggest libraries loaded in the storefront.

Using a (currently unreleased) tool to bundle a stock Luma home page, jQueryUI is 227kb of the shared bundle (unminified), even though only 11% of that is used on the homepage. The total bundle in my test was 774kb, so almost 1/3 of the bundle is jQuery UI.

We currently have a copy of jQueryUI in core with all these plugins in a single file:

jquery.ui.core.js
jquery.ui.widget.js
jquery.ui.mouse.js
jquery.ui.draggable.js
jquery.ui.droppable.js
jquery.ui.resizable.js
jquery.ui.selectable.js
jquery.ui.sortable.js
jquery.ui.effect.js
jquery.ui.accordion.js
jquery.ui.autocomplete.js
jquery.ui.button.js
jquery.ui.datepicker.js
jquery.ui.dialog.js
jquery.ui.effect-blind.js
jquery.ui.effect-bounce.js
jquery.ui.effect-clip.js
jquery.ui.effect-drop.js
jquery.ui.effect-explode.js
jquery.ui.effect-fade.js
jquery.ui.effect-fold.js
jquery.ui.effect-highlight.js
jquery.ui.effect-pulsate.js
jquery.ui.effect-scale.js
jquery.ui.effect-shake.js
jquery.ui.effect-slide.js
jquery.ui.effect-transfer.js
jquery.ui.menu.js
jquery.ui.position.js
jquery.ui.progressbar.js
jquery.ui.slider.js
jquery.ui.spinner.js
jquery.ui.tabs.js
jquery.ui.tooltip.js

We should modularize our usage this library, while still maintaining backwards-compatibility.

Game Plan

Change the jQueryUI monolith build in Magento 2 to be all individual files (someone ran a tool to bundle them all when they were committed years ago, so basically undo the bundling part). Make sure it's the same version of jQueryUI that Magento already uses
Add shim config for each jQueryUI plugin if it depends on other jQueryUI plugins
Create a file (example: jquery-ui-compat.js), and make it import every jQueryUI widget
In requirejs-config.js in core, change the paths config for jquery/ui to point to jquery-ui-compat
In every place in core JS where jquery/ui is used, change the dependency list to import every jQuery UI plugin that is used by the module.

Example

I created a hacky POC that does this for the Luma homepage with Magento stock data. Here were my results:

Repo with POC

m2-assistant[bot] commented 5 years ago

Hi @DrewML. Thank you for your report. To help us process this issue please make sure that you provided the following information:

[x] Summary of the issue
[x] Information on your environment
[x] Steps to reproduce
[ ] Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento-engcom-team give me 2.3-develop instance - upcoming 2.3.x release

For more details, please, review the Magento Contributor Assistant documentation.

@DrewML do you confirm that you were able to reproduce the issue on vanilla Magento instance following steps to reproduce?

[x] yes
[ ] no

Join Magento Community Engineering Slack and ask your questions in #github channel.

DrewML commented 5 years ago

Internal tracking ticket: MC-16887

Vinai commented 5 years ago

This is great 👍🏻

DrewML commented 5 years ago

@adifucan has picked this up internally. Seeing good results so far 👍, and will be backwards-compatible

adifucan commented 5 years ago

jQuery UI library was split into separate components for frontend area. All core modules that rely on entire jQuery UI library have had their dependency lists modified to import just the jQuery UI components they need. Adminhtml modules still rely on entire jQuery UI library.

So, if you never declared dependency to jQuery UI library in your JS component but used it, you may get an error. You should define dependency to separate jQuery UI component as there is no jQuery UI library on storefront anymore. If you declare entire jQuery UI library as a dependency in your JS components, it will work but you won't see performance improvements. Moreover you may see performance degradation as together with separate jQuery UI components all jQuery UI library will be loaded either.

Splitting jQuery UI library into small components result in the following performance improvements for frontend:

JS files count and requests count increased. It's expected behavior because instead of 1 jQuery UI library a bunch of widget files started to load.
JS files size and fully loaded time decreased for almost all scenarios.
Google PageSpeed Insights score improved.
Now on home page there are 88 KB of jQuery UI components transferred instead of 125 KB of entire jQuery UI library transferred.

m2-assistant[bot] commented 4 years ago

Hi @engcom-Charlie. Thank you for working on this issue. In order to make sure that issue has enough information and ready for development, please read and check the following instruction: :point_down:

[ ] 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).
Details
If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.
[ ] 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.
[ ] 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.
[ ] 4. Verify that the issue is reproducible on 2.3-develop branch
Details
- Add the comment @magento give me 2.3-develop instance to deploy test instance on Magento infrastructure.
- If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
- If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!
[ ] 5. Add label Issue: Confirmed once verification is complete.
[ ] 6. Make sure that automatic system confirms that report has been added to the backlog.

serge-kaimin commented 4 years ago

Decomposing javascript library can cause missing cache on modern browsers, while missing cache would affect performance on each page load.

Chromium's engine has a minimum size for code caches, currently set to 1 KiB of source code. This means that smaller scripts are not cached at all. You may want to consider merging small javascript code together to bundles so that they exceed the minimum code size, as well as benefiting from generally reducing script overheads.

reference: https://cs.chromium.org/chromium/src/third_party/blink/renderer/bindings/core/v8/v8_code_cache.cc?l=91&rcl=2f81d000fdb5331121cba7ff81dfaaec25b520a5

adifucan commented 4 years ago

@serge-kaimin Thanks for your comment. Most of jquery ui components are more than 1KiB. Anyway I guess missing cache on modern browsers may affect developer mode mostly as on production we recommend to bundle JS.

serge-kaimin commented 4 years ago

I just want to notice once more time, that its not true for almost scenarios - "JS files size and fully loaded time decreased for almost all scenarios." Most of jquery ui components has size less than 1.8Kb, when js-uglified it would be less than 1Kb, then miss the browser's cache. That allmost half of JQuery UI scripts would be loaded on each new page reload or category or product visit. Its realy bad scenario for all 3g mobile visitors.

bundlegento has just 7 stars, now. Did you tried to bundle that way or even cloned it already? I believe almost all from the box solutions to bundle magento's javascript is just waisting the time. Almost magento in production sites are missing cache now, because hundreds of small not cached instances. You can see it via webpagetest's request map or waterfall view. Just webpagetest almost any of large Magento sites to see the issue.

To improve Google PageSpeed just load almost of libraries async/defer way and/or pull it from CDN (browser's cache). Set up tls/1.3, implement static brotli compression to save 10-20% of size. There is many ways to get better PageSpeed rank but not decomposition of jQuery UI. Do you know that not all Magento owners migrated to http/2?

At least good idea is to analyze existing magento's implementations. HTTP Web Archives are open for data analyzis, just few querries required.

Even you finally decided to decompose, good idea is to analyze which effects and widgets could be bundled out of the box by smart way!

Remember about minimum 1Kb cache size limit all the time when you try to decompose!

DrewML commented 4 years ago

Thanks for the feedback @serge-kaimin - appreciate it.

Chromium's engine has a minimum size for code caches, currently set to 1 KiB of source code. This means that smaller scripts are not cached at all

This threshold is set for a good reason: caching has a cost. In these cases it's cheaper for v8 to do compilation than it is to cache the result. In a hypothetical scenario, if a 1 KiB asset takes 50ms to compile or 60ms to use caching overhead, it would make sense not to hit the cache.

The recommendation of combining everything adds compilation time on the first render, instead of amortizing the cost over several pages.

Keep in mind too that this is a heuristic that has changed (and will again), and is not consistent between browser vendors. Chasing engine heuristics isn't something I'd be super interested in.

bundlegento has just 7 stars, now. Did you tried to bundle that way or even cloned it already

@adifucan is very familiar - he helped build it :)

To improve Google PageSpeed just load almost of libraries async/defer way and/or pull it from CDN (browser's cache)

re: async/deferred loading, this is already how baler bundling works.

re: pulling from a CDN, this only makes the problem worse. That adds an additional DNS lookup + TCP connection + TLS handshake to the equation. With Chrome recently adopting partitioning of the HTTP cache (already done in webkit), you will never get a CDN cache hit unless the user has already been to your site (and then it would have been cached without the CDN).

Do you know that not all Magento owners migrated to http/2

This is not something we should be optimizing for. If a store cares about performance, they should be using HTTP/2. If there are stores that haven't made the switch, we should be looking for ways to encourage them.

Almost magento in production sites are missing cache now, because hundreds of small not cached instances. You can see it via webpagetest's request map or waterfall view. Just webpagetest almost any of large Magento sites to see the issue.

To the best of my knowledge, WebPageTest only exposes network cache hits/misses in these views, and does not expose anything about compilation cache (you have to drop into Chromium's trace viewer). Can you clarify or provide some screenshots about what cache metrics you're referring to?

It's also worth noting that the v8 developers have explicit feedback on their blog about optimizing for engine code caching:

Ideally, the best thing you as a JS developer can do to improve code caching is “nothing”. This actually means two things: passively doing nothing, and actively doing nothing.

Code caching is, at the end of the day, a browser implementation detail; a heuristic-based data/space trade-off performance optimization, whose implementation and heuristics can (and do!) change regularly. We, as V8 engineers, do our best to make these heuristics work for everyone in the evolving web, and over-optimising for the current code caching implementation details may cause disappointment after a few releases, when those details change. In addition, other JavaScript engines are likely to have different heuristics for their code caching implementation. So in many ways, our best advice for getting code cached is like our advice for writing JS: write clean idiomatic code, and we’ll do our best to optimise how we cache it.

In addition to passively doing nothing, you should also try your best to actively do nothing. Any form of caching is inherently dependent on things not changing, thus doing nothing is the best way of allowing cached data to stay cached. There are a couple of ways you can actively do nothing.

serge-kaimin commented 4 years ago

There is latest jQuerry-UI, minified, and compressed by gzip and brotli: 520714 jquery-ui.js 253668 jquery-ui.min.js2 67940 jquery-ui.min.js.gz 57187 jquery-ui.min.js.br

Lets assume that almost of Magento stores are using gzip now, not brotli. If you will decompose one file to many small instances, I suppose on many small files there would be bigger overhead of gzip's dictionary than amount of saved network resources.

If you just implement static compression using brotli, performance budget almost would be the same but no cache issues. How performance budget would change If to implement brotli + guetzli, try comparet with decompose of 2-3 libraries on to small insances?

Chasing engine heuristics isn't something I'd be super interested in. Not good news for me. As I know Andy and his team is working closely with largest like facebook, tweeter, and instagram, and others even if G compete on some markets. They are trying to improve continuoulsy core of the web browser engine in terms of javascript processing, cache optimization, and even lazy loading, all staff just to improve the web and customer experience.

By climbing on PWA and react.js wagoon, you will see on how speed of Magento would be better just because web browsers are continuolsy improved. They do improve because they do evaluate and work together on one goal to make web better, but not because they are just so smart guys. Better to be close to the machinist and know where the train is going to and even try to insist and propose new pass as well.

what cache metrics you're referring to? just a file size, if its less than 1 kb, it would never hit chrome's cache.

I suppose V8 can change cache strategy each month but 1kb memory limit would be there after EOL for Magento Knockout...

re: pulling from a CDN, this only makes the problem worse. dns prefetch, preload, and many other solutions can work just fine. all we will use HTTP/3 soon, thanks again to g. Cache hits and real performance better to be tested using boomerang for specific libraries, cdn, region, and etc. Better if you will operate such statics based on big regional data even collected from HTTP archives than just assumptions and previous experience. That is my point - that your assumption are not correct: files size and fully loaded time decreased for almost all scenarios and PageSpeed Insights score improved. There is important issues behind just home page visit and page speed score improved not because jquerry UI decomposed but UI components loaded after DOM processing.

I suppose the future of choosing location from where to load specific libraries or even media files would be not on the frontend of application but middleware of edge-proxy/front-proxy like envoy controlled by istio. You don't like CDN, that's ok, smart network guys on edge server would decide depending on statistical analysis, performance monitoring of app mesh, date, device, and origin of IP address which library to load to specific visitor, to specific device, or even to specific monitor of some specific device, and replace url on the fly by the CDN's url closest to the visitor. Should it be integrated AMP-network-mesh-+PWA? Will see..

p.s. thanks to pointing on baler, will take a look on it.

sdzhepa commented 4 years ago

Hello @DrewML @adifucan @serge-kaimin

As I can see all planed for on this Issue has been done in the scope of internal JIRA ticket MC-16887 and merged into 2.3-develop All related commits: https://github.com/magento/magento2/search?q=MC-17868&type=Commits

I suppose we can close this issue. Please let me know if I missed something and we need to reopen it or/and move into Magento Feature Request public repo.

monotheist commented 4 years ago

Preconditions (*)
1. Any version of Magento 2
Steps to reproduce (*)
1. Not a bug
Expected result (*)
1. I expect us to only load JavaScript that is used on a particular page.
Actual result (*)
1. We load several hundred kilobytes of dead code
Further Details

jQuery UI is one of the biggest libraries loaded in the storefront.

Using a (currently unreleased) tool to bundle a stock Luma home page, jQueryUI is 227kb of the shared bundle (unminified), even though only 11% of that is used on the homepage. The total bundle in my test was 774kb, so almost 1/3 of the bundle is jQuery UI.

We currently have a copy of jQueryUI in core with all these plugins in a single file:
jquery.ui.core.js
jquery.ui.widget.js
jquery.ui.mouse.js
jquery.ui.draggable.js
jquery.ui.droppable.js
jquery.ui.resizable.js
jquery.ui.selectable.js
jquery.ui.sortable.js
jquery.ui.effect.js
jquery.ui.accordion.js
jquery.ui.autocomplete.js
jquery.ui.button.js
jquery.ui.datepicker.js
jquery.ui.dialog.js
jquery.ui.effect-blind.js
jquery.ui.effect-bounce.js
jquery.ui.effect-clip.js
jquery.ui.effect-drop.js
jquery.ui.effect-explode.js
jquery.ui.effect-fade.js
jquery.ui.effect-fold.js
jquery.ui.effect-highlight.js
jquery.ui.effect-pulsate.js
jquery.ui.effect-scale.js
jquery.ui.effect-shake.js
jquery.ui.effect-slide.js
jquery.ui.effect-transfer.js
jquery.ui.menu.js
jquery.ui.position.js
jquery.ui.progressbar.js
jquery.ui.slider.js
jquery.ui.spinner.js
jquery.ui.tabs.js
jquery.ui.tooltip.js
We should modularize our usage this library, while still maintaining backwards-compatibility.

Game Plan
1. Change the `jQueryUI` monolith build in Magento 2 to be all individual files (someone ran a tool to bundle them all when they were committed years ago, so basically undo the bundling part). Make sure it's the same version of jQueryUI that Magento already uses

2. Add shim config for each jQueryUI plugin if it depends on other jQueryUI plugins

3. Create a file (example: `jquery-ui-compat.js`), and make it import _every_ jQueryUI widget

4. In `requirejs-config.js` in core, change the paths config for `jquery/ui` to point to `jquery-ui-compat`

5. In every place in core JS where `jquery/ui` is used, change the dependency list to import every jQuery UI plugin that is used by the module.
Example

I created a hacky POC that does this for the Luma homepage with Magento stock data. Here were my results:

Repo with POC

Tan

Preconditions (*)
1. Any version of Magento 2
Steps to reproduce (*)
1. Not a bug
Expected result (*)
1. I expect us to only load JavaScript that is used on a particular page.
Actual result (*)
1. We load several hundred kilobytes of dead code
Further Details

jQuery UI is one of the biggest libraries loaded in the storefront.

Using a (currently unreleased) tool to bundle a stock Luma home page, jQueryUI is 227kb of the shared bundle (unminified), even though only 11% of that is used on the homepage. The total bundle in my test was 774kb, so almost 1/3 of the bundle is jQuery UI.

We currently have a copy of jQueryUI in core with all these plugins in a single file:
jquery.ui.core.js
jquery.ui.widget.js
jquery.ui.mouse.js
jquery.ui.draggable.js
jquery.ui.droppable.js
jquery.ui.resizable.js
jquery.ui.selectable.js
jquery.ui.sortable.js
jquery.ui.effect.js
jquery.ui.accordion.js
jquery.ui.autocomplete.js
jquery.ui.button.js
jquery.ui.datepicker.js
jquery.ui.dialog.js
jquery.ui.effect-blind.js
jquery.ui.effect-bounce.js
jquery.ui.effect-clip.js
jquery.ui.effect-drop.js
jquery.ui.effect-explode.js
jquery.ui.effect-fade.js
jquery.ui.effect-fold.js
jquery.ui.effect-highlight.js
jquery.ui.effect-pulsate.js
jquery.ui.effect-scale.js
jquery.ui.effect-shake.js
jquery.ui.effect-slide.js
jquery.ui.effect-transfer.js
jquery.ui.menu.js
jquery.ui.position.js
jquery.ui.progressbar.js
jquery.ui.slider.js
jquery.ui.spinner.js
jquery.ui.tabs.js
jquery.ui.tooltip.js
We should modularize our usage this library, while still maintaining backwards-compatibility.

Game Plan
1. Change the `jQueryUI` monolith build in Magento 2 to be all individual files (someone ran a tool to bundle them all when they were committed years ago, so basically undo the bundling part). Make sure it's the same version of jQueryUI that Magento already uses

2. Add shim config for each jQueryUI plugin if it depends on other jQueryUI plugins

3. Create a file (example: `jquery-ui-compat.js`), and make it import _every_ jQueryUI widget

4. In `requirejs-config.js` in core, change the paths config for `jquery/ui` to point to `jquery-ui-compat`

5. In every place in core JS where `jquery/ui` is used, change the dependency list to import every jQuery UI plugin that is used by the module.
Example

I created a hacky POC that does this for the Luma homepage with Magento stock data. Here were my results:

Repo with POC

Thank you. Which plugins from these should be used to achieve smartheader, stickyheader and expandingsearch functionalities? Any help would be highly appreciated.

G4briel117 commented 1 year ago

This is great 🤝🏾

magento / magento2