search

ampproject / amphtml

The AMP web component framework.
https://amp.dev
Apache License 2.0
14.89k stars 3.89k forks source link

lazy load AMP SCRIPTS #33689

Open hmarsili opened 3 years ago

hmarsili commented 3 years ago

Describe the new feature or change to an existing feature you'd like to see

Currently, when loading a complex AMP page like news a article, 8 to 15 AMP scripts are loaded, even those which are not required on the first scroll and it might never been use for users. For example, Facebook Comments scripts on the bottom of the page. We know AMP SCRIPTS are async, nevertheless fewer scripts seems to increase overall performance and specially LCP for the page.

In our use case, at least 7 scripts are not needed until the user start scrolling. If the users does not consume at least 25% of the page, those scripts are not needed.

Describe alternatives you've considered

Provide a clear and concise description of any alternative solutions or features you've considered.

Additional context

Screen Shot 2021-04-07 at 16 55 16

kristoferbaxter commented 3 years ago

Apologies for the delay, am running tests to validate the idea.

kristoferbaxter commented 3 years ago

Am out of the office today, but here are some initial results and will continue investigating later today or tomorrow.

This test entirely removes all additional scripts beyond AMP's runtime, and causes LCP to fire sooner as well as CLS to occur sooner due to layout issues.

Screen Shot 2021-04-12 at 11 22 08 AM

Next I'll create a small loader prototype for lazy loading, to see what part of that potential maximal impact can realistically be achieved (my current theory is less than 10%).

kristoferbaxter commented 3 years ago

Here's an example with the minimal components needed for the first viewport of this document (the remainder are lazy loaded by a small inline loader attached to each amp- component.

Screen Shot 2021-04-12 at 2 39 43 PM

This shows the value of only loading the first viewport versus the entire documents components as async JavaScript to be very minimal in this example.

My take after trying this for a short period is we should put more focus on reducing the size of the blocking v0.js/v0.mjs payload versus the size of async extensions. However, this is a good candidate for revisiting after the larger win from removing logic and complexity from the AMP runtime.

hmarsili commented 3 years ago

Am out of the office today, but here are some initial results and will continue investigating later today or tomorrow.

This test entirely removes all additional scripts beyond AMP's runtime, and causes LCP to fire sooner as well as CLS to occur sooner due to layout issues.

Screen Shot 2021-04-12 at 11 22 08 AM

Next I'll create a small loader prototype for lazy loading, to see what part of that potential maximal impact can realistically be achieved (my current theory is less than 10%).

Hi, I think this is very promising. Maybe the LS occurs because a script needed on the first viewport was removed? I our test we saw a performance improvement on LCP while minimizing the amount of scripts. Plainly, a page with 15 AMP scripts does not perform the same as one with 4. And that we think is related to the scripts being fired up on start.

kristoferbaxter commented 3 years ago

The issue is the example sets require many more than 4 extensions when tested so far, can you provide an example where 15 => 4 would not break the first viewport or analytics?

Also, the second post is the one to look at for this example, since it includes the necessary scripts for the first viewport. The example quoted was the maximal potential impact of this approach for lazy loading all scripts (which would break the first viewports functionality and analytics).

Edits: Grammar mistakes.

hmarsili commented 3 years ago

Let me check that with my team and get back to you with that. I think we have a couple but not sure if includes GA. Anyway, the my point is more scripts fired, longer the LCP. I think that might be particulate true for low end mobile devices. In our tests we emulated a MOTO G4 / fast 3G.

kristoferbaxter commented 3 years ago

I think that might be particulate true for low end mobile devices. In our tests we emulated a MOTO G4 / fast 3G.

All of these tests are against a Moto G4 with a 3G connection.

Anyway, the my point is more scripts fired, longer the LCP.

The number of requests and amount of data transferred does matter, but how those bytes are used is also important to consider. If the page is not usable (either at the scrolled position on reload, or by default in the first viewport) then the tradeoff to lazy load subsections of the page means the user is also not getting a good experience.

My point I am trying to make in these examples is that the absolutely necessary scripts during initial execution and lazy loading only the parts of the DOM further down the page doesn't provide significant benefit because the size of the optional lazy loaded content is so small relative to the minimum required. As a result, lazy loading the small amount doesn't pay much benefit for many scenarios and we should instead focus on making the default required elements significantly smaller instead. (via updated components and runtime)

Also totally possible I am missing something about the recommendation here!

hmarsili commented 3 years ago

@kristoferbaxter for a standard news site (articles) like xoxo.news or koko.news we need only 5 scripts for the first two scrolls.

kristoferbaxter commented 3 years ago

Running another test to confirm first set of results.

kristoferbaxter commented 3 years ago

I've run another set of tests for the supplied article and it confirms the minimal impact of this proposed change.

Moving to lazy loading of extensions not in the first viewport would save .06s off the LCP timing on a 3G connection in this lab environment. The size of AMP extensions (and their execution) is not a material member of the initial performance of this document or similar ones.

Default

Screen Shot 2021-04-14 at 9 50 59 AM

Minimal Extensions (just the ones listed in https://github.com/ampproject/amphtml/issues/33689#issuecomment-818912531)

Screen Shot 2021-04-14 at 9 53 22 AM

No Extensions

Screen Shot 2021-04-14 at 9 55 18 AM

I'll update the results with a different pathway (removing the larger runtime for a more svelte one) when the test completes.

kristoferbaxter commented 3 years ago

For the next set of images I've switched to Slow3G and removed the additional critical DNS resolution for the image (like it would be in the Optimized version of the document by default).

Default

Screen Shot 2021-04-15 at 10 00 21 AM

Minimal

Screen Shot 2021-04-15 at 10 00 46 AM

No Extensions

Screen Shot 2021-04-15 at 10 01 17 AM

The reason for little to no impact to LCP, CLS, or TBT is because the document is able to trigger LCP without needing JS. CLS is consistent as extensions are lazyloaded and potentially causing layout shift, and TBT remains relatively constant since the amount of work is amortized across page load.

samouri commented 3 years ago

nevertheless fewer scripts seems to increase overall performance and specially LCP for the page.

@hmarsili: can you speak more to the performance experiments you ran and the LCP win you were able to see by delaying the load of JS scripts? Were they in line with what @kristoferbaxter measured/estimates (10%) or did they indicate a greater win?

FWIW, I just measured the page's LCP locally with JS completely disabled vs. enabled over 10 runs with "Fast 3G" and CPU 4x Throttle and saw a 25% decrease:

This further supports 10% as a ballpark.

hmarsili commented 3 years ago

Hi all, we think is about 15%/20% decrease, which is a lot in the context of CWV. In our testing, one outcome was particularly good. Removing all the none essential extensions promoted POOR URLs to NEED FOR IMPROVEMENT and NEED FOR IMPROVEMENT to GOOD, in our test case xoxo.news. The point is, with a target LCP 2.4 or less, a 0.2 or 0.3 can make the difference between a GOOD URL or not, specially in slow bandwidth countries.

samouri commented 3 years ago

@hmarsili: can you provide information on how to reproduce your tests that demonstrated the 15-20% decrease. We need to be able to replicate these experiments on our end. At this point we've performed what experiments we could think of, and haven't seen anything to indicate such a drastic improvement is likely. To be clear: I would love to be wrong.

The point is, with a target LCP 2.4 or less, a 0.2 or 0.3 can make the difference between a GOOD URL or not, specially in slow bandwidth countries.

One of AMP's highest priorities is to be as performant as possible. We have various initiatives we are pursuing to accomplish that. The way we triage what to spend time on is by comparing the impact and cost-to-implement of the various ideas. If you provide us with reproduction steps that show a 15-20% decrease, it will help us prioritize this idea against our other initiatives.

hmarsili commented 3 years ago

Let me create a couple of TEST pages with my team and we will share the results. To be 100% honest, test results sometimes are erratic. You get A A A A and then the other day is all B B B B :)

+Martin Juzefiszyn @.***> can we create test pages for the same site with 5 extensions and with the complete set?

Saludos, Hernán.

[image: Logo] Hernán Marsili Director, TECHNOLOGY FOR SOLUTIONS. Tel. +54 9 11 4945 2272 | [image: Twitter icon] https://twitter.com/cmsmedios [image: LinkedIn icon] https://www.linkedin.com/company/technology-for-solutions/ [image: Banner] http://www.cms-medios.com/ El contenido de este correo electrónico es confidencial y está destinado al destinatario especificado solo en el mensaje. Está estrictamente prohibido compartir cualquier parte de este mensaje con terceros, sin el consentimiento por escrito del remitente.

On Mon, Apr 19, 2021 at 3:03 PM Jake Fried @.***> wrote:

@hmarsili https://github.com/hmarsili: can you provide information on how to reproduce your tests that demonstrated the 15-20% decrease. We need to be able to replicate these experiments on our end. At this point we've performed what experiments we could think of, and haven't seen anything to indicate such a drastic improvement is likely. To be clear: I would love to be wrong.

The point is, with a target LCP 2.4 or less, a 0.2 or 0.3 can make the difference between a GOOD URL or not, specially in slow bandwidth countries.

One of AMP's highest priorities is to be as performant as possible. We have various initiatives we are pursuing to accomplish that. The way we triage what to spend time on is by comparing the impact and cost-to-implement of the various ideas. If you provide us with reproduction steps that show a 15-20% decrease, it will help us prioritize this idea against our other initiatives.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ampproject/amphtml/issues/33689#issuecomment-822666004, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAH4EPGAGG2CPQ7HB2YEOADTJRWAPANCNFSM42RN77PQ .

hmarsili commented 3 years ago

Hi!

I share the tests that we have created to share the results later.

[image: Logo] Martín Juzefiszyn Especialista en Soporte, TECHNOLOGY FOR SOLUTIONS. Tel. +54 11 4711 8999 | Cel. +54 9 341 201 1360 | [image: Twitter icon] https://twitter.com/cmsmedios [image: LinkedIn icon] https://www.linkedin.com/company/technology-for-solutions/ [image: Banner] http://www.cms-medios.com/ El contenido de este correo electrónico es confidencial y está destinado al destinatario especificado solo en el mensaje. Está estrictamente prohibido compartir cualquier parte de este mensaje con terceros, sin el consentimiento por escrito del remitente.

El lun, 19 de abr. de 2021 a la(s) 17:16, Hernán Marsili @.***) escribió:

Let me create a couple of TEST pages with my team and we will share the results. To be 100% honest, test results sometimes are erratic. You get A A A A and then the other day is all B B B B :)

+Martin Juzefiszyn @.***> can we create test pages for the same site with 5 extensions and with the complete set?

Saludos, Hernán.

[image: Logo] Hernán Marsili Director, TECHNOLOGY FOR SOLUTIONS. Tel. +54 9 11 4945 2272 | [image: Twitter icon] https://twitter.com/cmsmedios [image: LinkedIn icon] https://www.linkedin.com/company/technology-for-solutions/ [image: Banner] http://www.cms-medios.com/ El contenido de este correo electrónico es confidencial y está destinado al destinatario especificado solo en el mensaje. Está estrictamente prohibido compartir cualquier parte de este mensaje con terceros, sin el consentimiento por escrito del remitente.

On Mon, Apr 19, 2021 at 3:03 PM Jake Fried @.***> wrote:

@hmarsili https://github.com/hmarsili: can you provide information on how to reproduce your tests that demonstrated the 15-20% decrease. We need to be able to replicate these experiments on our end. At this point we've performed what experiments we could think of, and haven't seen anything to indicate such a drastic improvement is likely. To be clear: I would love to be wrong.

The point is, with a target LCP 2.4 or less, a 0.2 or 0.3 can make the difference between a GOOD URL or not, specially in slow bandwidth countries.

One of AMP's highest priorities is to be as performant as possible. We have various initiatives we are pursuing to accomplish that. The way we triage what to spend time on is by comparing the impact and cost-to-implement of the various ideas. If you provide us with reproduction steps that show a 15-20% decrease, it will help us prioritize this idea against our other initiatives.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ampproject/amphtml/issues/33689#issuecomment-822666004, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAH4EPGAGG2CPQ7HB2YEOADTJRWAPANCNFSM42RN77PQ .

kristoferbaxter commented 3 years ago

Thanks for making the samples! I'll run tests against these.