Add a memory/size audit

[natorion]

Summary

We should provide a way in Lighthouse to audit memory/size of a web page.

Questions answered

Description of audit and audit category (please include pass/fail states, and how it might be measured)

We should provide a new audit category aimed at "Memory" (or "Size", which might fit even better) e.g. see the mock below

In the first stage we should aim to highlight the full memory consumption. Currently Chrome's main memory tooling only surfaces around 50 % of the actual memory consumed. We have the total memory consumption of all chrome components in the mem-infra trace though.

Stage 1

What number do we want to surface:

Live full peak memory consumption:

• Private dirty c.f. performance.memory proposal
• JavaScript heap size and mention that this is the memory that is reported by performance.memory DOM elements
• Everything else on the renderer
• Theoretically we could measure live memory during the distinct loading phases like "First time to interactive", I am not sure if this would add any value though.
• Fail state: If peak memory is too high to be opened on 512M and 1024M Android devices (Android Go class), fail and mention this
• Data: We should already be able to collect the relevant numbers via tracing (mem-infra). This will affect performance though.

  JavaScript code coverage:

• Mention how much of the shipped code is actually executed
• Fail state: Anything below 50 % (?)
• There are valid use cases like user interaction handlers that are preloaded.
• It will heavily depend on the use case/app.
• Use https://medium.com/dev-channel/the-cost-of-javascript-84009f51e99e and other, similar sources to come up with a better percentage
• Data: V8/DevTools already supports reporting code coverage. This will affect performance though.

Stage 2

What number do we want to surface:

Impact of cross origin iframes

• When Site isolation is enabled, cross origin iframes will be put into a separate process. There is memory overhead dependant on the platform (Android, Windows, Mac) for each new process. We should show an estimate how much memory is used for using cross-origin iframes (typically used in Ads).
• This should be shown in the Live peak memory consumption category of Stage 1 and include a similar fail state.
• Data: Tracing (mem-infra) already provides this data
• If we have case studies at that point tying memory consumption to business metrics, we should use them here too.

Hacked-together mock:

Explanation of how it’s different from other audits

Memory/Size is currently not handled in Lighthouse.

What % of developers/pages will this impact (estimates OK, data points preferred)

When Lifecycle API or similar mechanisms are shipped, all of them.

How is the new audit making a better web for end users? (data points preferred)

Less consumption of precious system resources (RAM). Less trashing and emergency GCs.

What is the resourcing situation (who will create the audits, maintain the audits, and write/maintain the documentation)

Let's talk :-).

Do you envision this audit in the Lighthouse report or the full config in the CLI? If in the report, which section?

This should be in the default Lighthouse report.

How much support is needed from the Lighthouse team?

Let's talk :-).

Any other links or documentation that we should check out?

https://docs.google.com/presentation/d/14uV5jrJ0aPs0Hd0Ehu3JPV8IBGc3U8gU6daLAqj6NrM/edit#slide=id.p and other relevant, Google-internal docs.

[vinamratasingal-zz]

Hey Michael- thanks for submitting this. We're heads down on I/O, let's chat after about this proposal.

[vinamratasingal-zz]

Hey Michael- thanks for putting together this detailed proposal and for your patience! The team and I had a chance to sync up and we have a couple pieces of feedback:

• We'd like to see some more data points around how you're planning on getting developers to care about memory, and activate the ecosystem. My understanding is that there's a case study in the works- we'd love to see the output of the study to see how much impact memory has on improving metrics developers care about.
• I think there were also some technical questions around how Lighthouse might be able to report on these metrics given the underlying implementation of LH- but I think we can address these laters.
• Any reason why these audits shouldn't live in DevTools rather than LH?

[natorion]

There is indeed a case study in the works. We cannot make the numbers of the currently running study public though.

Regarding the technical questions, what specific concerns do you have?

I guess the audits can also live in DevTools. My impression was that audits should go to LH though? My impression is that if I open DevTools I already know that there is a problem I want to investigate and LH is a handling the step before that: Highlighting that there is something to fix. I also heard from web developers that they use LH in their CI to test for 'compliance', something that will not automatically work with DevTools.

[natorion]

Friendly ping on this.

[natorion]

@paulirish

[doteric]

Sounds like a great idea, by any chance is there any plan to bring this back up? Memory leaks seem to be more often the case these days on complex apps for some reason...
Preferably it would be nice if this was available in user flows as memory is often the problem after a certain amount of user interactions rather than on page load.

CC @adamraine

[adamraine]

We definitely could offer some memory statistics and heap snapshots, but with the way Lighthouse has evolved I don't think a timeline view of memory makes sense anymore.

[doteric]

Thanks @adamraine for the quick feedback, Could you please elaborate on the "I don't think a timeline view of memory makes sense anymore" part please? Not sure what you mean by that exactly? Would appreciate it

[adamraine]

I was referring to some sort of graph that shows the memory usage over time. This would be hard to visualize due to how Lighthouse throttles the page by default.

[doteric]

Ah alright, totally a graph would not be needed but data points of memory being used on certain stages would be nice. It would be useful on userflows especially to see the amount of memory between certain actions 😃
Thanks for the replies Adam btw 💪