CLI option to reset cache but not storage APIs

[gruppjo]

Hi everyone :)

I've started using lighthouse for one of our large scale Angular 2/4 apps and it's working really nicely. Thank you!

The app has authentication, cookie-layers etc... So I needed to run lighthouse with chrome-debug and use --disable-storage-reset which does [link]:

Disable clearing the browser cache and other storage APIs before a run

Incidentally I was hoping that --disable-storage-reset would only disable a reset for Cookies, localstorage and the other APIs and that the cache on the other hand would still be reset before every run.

That behavior is the one I'm most interested in since it's a very common case that users will access new areas (no content in browser cache) of the app but will have already logged in. Currently I can't test that.

So I'd recommend another CLI option that does just that.

What do you think?

Best, Jon

[patrickhulce]

For a little more insight into the issue, what exact steps do you take to run Lighthouse today and which pages are you hoping to audit with a clean cache?

Current breakdown as I understand it, but perhaps your case differs

1. Login page, no auth needed (should be auditable with clean cache today)
2. Protected landing page, auth needed (not easily auditable with clean cache today)
3. Protected nested page, auth needed (should still be auditable with the cache that a user is likely to have after just logging in)

We'd also be interested to have your input in #1418 for any patterns in your use case that might not have been identified yet :)

[gruppjo]

Hi @patrickhulce,

Recurring visits with empty cache

Your breakdown is pretty much what I'm trying to do :) One one addition:

• I'm especially interested in recurring visits, on which it's likely that the user is still authenticated via cookies but the browser cache is not present anymore (e.g. browser was shut down or the site is visited again after some days of inactivity). That's where --disable-all-but-cache would come in handy.

Current manual workflow

Currently I do the following:

1. run chrome debug
2. go to the website, accept the cookie agreement (persisted in localstorage), perform a login (persisted via cookie)
3. start lighthouse via the CLI lighthouse ... --disable-storage-reset

Ideal automated workflow

In order to automate this process, setting the localstorage and cookie values somehow as recommended in https://github.com/GoogleChrome/lighthouse/issues/1418 would be necessary. This is especially the case, since on mobile the cookie agreement renders fullscreen, which renders the lighthouse report's screenshots rather pointless.

Should I comment that localstorage part in https://github.com/GoogleChrome/lighthouse/issues/1418? It's feels more like a separate issue to me.

Best, and thanks :)

[wardpeet]

how do we feel about --disable-storage-reset=<comma seperated list>

--disable-storage-reset="localstorage,indexdb,websql,cookies,cache,appcache"

[patrickhulce]

Oooh I like that @wardpeet, might be a bit confusing what the intention is though we'll have to make it clear that you're asking to keep the storages that you list there. Double negative getting confusing 😆

@gruppjo for complicated cases where you need to setup multiple storages in a particular way I'm not sure we'll get to a place where Lighthouse will have that functionality baked in, your best bet for now might be creating a custom gatherer that sets up your requirements in the beforePass (which runs before we try to load the page).

[wardpeet]

@patrickhulce we could also do the other way around --enable-storage-reset="localstorage,indexdb,websql,cookies,cache,appcache" with those as a default but might be more confusing I don't know 🤔

[patrickhulce]

The more common case I imagine will be "just keep localStorage" or "just keep cookies", so I think your first proposal makes sense just might need some explaining

[joshwiens]

@wardpeet - On by default & explicitly disabling certain storage reset options feels less confusing given the default of nuking everything out of the box, which in most cases is completely reasonable when benchmarking performance.

I'd say making the opt-out an explicit action would be the right way to go.

[wardpeet]

so we should probably go for --disable-storage-reset localstorage indexdb websql cookies cache appcache" using yargs array

[paulirish]

FWIW we prefer the API proposed here: https://github.com/GoogleChrome/lighthouse/issues/2599#issuecomment-312005577

[connorjclark]

This is probably a good first contribution?

[timothyjoh]

I have also been looking for this feature (please clear my cache but not my localStorage) not only in the command line, but also using it programmatically in node.

For instance, the website I am testing has a page-gate, asking which country user is from and age of user (for compliance reasons for cough cough medicinal herbs...). In order to bypass this modal, which is launched on every public page of the site, I need to have 4 localStorage variables set in the browser.

I can successfully see it working when using the command line to launch chrome-debug like so chrome-debug https://www.greenishmedicinalherb.com, getting the port number in the terminal, and then using that in launching lighthouse like lighthouse https://www.greenishmedicinalherb.com --disable-storage-reset --port 58018 --view

Problem is that I am seeing a 30-40 point drop in performance from the page-gate modal, and I can't be completely sure it isn't due to cache. Hence I want to be able to have the solution above to the CLI.

Also I want to be able to perform this programmatically, because I wrote a script that performs this test across different hosting providers (heroku, netlify, AWS, etc) and wrote a script to test them a dozen-or-two times each and average all the scores (throwing out outliers). Having this test automated so we can reliably see averages, and have it run in our CI/CD process is pretty crucial to measuring the performance we all want to achieve.

I can see this is pretty old of an issue, but wondering if it is getting any traction?

[patrickhulce]

For programmatic setup prior to Lighthouse you're going to want to look at #3837 and #1418 @timothyjoh. tl;dr use puppeteer and/or a custom gatherer in Lighthouse.

It just hasn't seen enough interest from external parties to receive priority over other ongoing efforts, but we like the idea.

[timothyjoh]

Thanks for the quick response, and these potential solutions, I will experiment over the week and update if I find anything relevant.

[InfobyteTarun]

@patrickhulce I would like to ask that how to utilize this option i.e. "--disable-storage-reset" in node programming using node library of lighthouse!

[patrickhulce]

@InfobyteTarun you can use the flags argument of the lighthouse function or settings property of the Lighthouse config.

lighthouse(url, {disableStorageReset: true})
// OR
lighthouse(url, flags, {extends: 'lighthouse:default', settings: {disableStorageReset: true}})

// DISCLAIMER: untested and written from memory, refer to docs for actual working code

see https://github.com/GoogleChrome/lighthouse/tree/master/docs for full examples and details

[yazfield]

@connorjclark started tinkering with this, it was fairly simple to make it work. I got confused by the tests and didn't manage to find my way there. Would you have some time to help out?

[mainsmirnov]

Any news for this issue?