Ability to bypass network cache

[OrKoN]

CDP offers a command to disable the network cache for a given browsing context (s) https://chromedevtools.github.io/devtools-protocol/tot/Network/#method-setCacheDisabled

Puppeteer exposes it as https://pptr.dev/api/puppeteer.page.setcacheenabled/

cc @whimboo

[whimboo]

Note that at the moment the browsingContext.reload command has a ignoreCache argument, and browsingContext.navigate doesn't care about that itself.

So the question is if each command should get such a ignoreCache argument, or if it should be enabled/disabled for all commands based on single other new command.

CC @jgraham

[jgraham]

For loads originating from WebDriver, I would make it per-command myself, since that seems more flexible and less stateful.

But there's also the question of what happens to requests not originating from WebDriver itself. If we want to disable the cache for those (without doing some hideous network request interception thing to ensure they never hit the cache) we perhaps need the global state (or a less hideous way to intercept requests and mark them as bypassing the cache, but that might be quite high overhead).

[whimboo]

@OrKoN do you have some feedback into how CDP handles that and what exactly should be affected by that feature?

[OrKoN]

After a quick look:

1) setting cache to disabled, evicts all resources from memory cache within one renderer https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/loader/fetch/memory_cache.h;l=70;drc=67d90538f11c6b232dbfd716075db52aeb34fd15 2) The MemoryCache resources are also evicted on committing load 3) For outgoing network requests, the request mode is set to bypass cache or force cache hit. Not sure how many caches are there on the network layer. 4) CORS preflight requests are also forced when cache is disabled 5) Also it affects the media player loading

I think the expectation is that it applies to all requests in the scope, even if they are not originating from the driver.

[juliandescottes]

I think all devtools offer a "disable cache" feature, which for Firefox is scoped per top-level browsing context (ie doesn't impact other tabs, even if they also have devtools open). But for BiDi, maybe this would make sense as a command on the network domain, with an optional contexts argument to restrict it to only certain top level contexts?

[css-meeting-bot]

The Browser Testing and Tools Working Group just discussed Ability to bypass network cache.

The full IRC log of that discussion

<AutomatedTester_> topic: Ability to bypass network cache
<AutomatedTester_> github: https://github.com/w3c/webdriver-bidi/issues/582
<AutomatedTester_> orkon: we have been working on the network interception spec and we have realised that it is important to bypass the netwrok cache . If there is a cache then the network interception might not be hit
<jgraham1> q+
<jdescottes> I can confirm this is an issue for interception in FF as well
<AutomatedTester_> ... and I think there is a use case for disabling the cache to see how cold start takes
<AutomatedTester_> ack next
<AutomatedTester_> jgraham1: as jdescottes mentioned,. this is a issue in firefox too
<jdescottes> q+
<AutomatedTester_> ... we may need to wire this through the fetch spec so that makes sense
<AutomatedTester_> ... and we need to know at what levels and we do the disabling here. I expect there is a lot of spec work here and probably little implementation work here
<AutomatedTester_> q?
<AutomatedTester_> ack next
<AutomatedTester_> jdescottes: I wanted to clarify. It is an issue for firefox in some but not all caches.
<orkon> q+
<AutomatedTester_> ... in Firefox it is probably better to have users disable it before setting it
<AutomatedTester_> ack next
<AutomatedTester_> orkon: how do you handle the case where there things done by service workers. Can you intercept them?
<AutomatedTester_> jdescottes: we don't handle that and is an open issue
<AutomatedTester_> jgraham1: we would need to update the service worker spec here and our spec doesn't handle this scenario
<AutomatedTester_> q?
<AutomatedTester_> ... and just for clarity, we should handle this scenario but it's not been a priority so far
<AutomatedTester_> orkon: it doesn't work in CDP either so it's a lot of effort
<AutomatedTester_> q?

[OrKoN]

PR https://github.com/w3c/webdriver-bidi/pull/721

[OrKoN]

I believe this can be closed since https://w3c.github.io/webdriver-bidi/#command-network-setCacheBehavior provides the required functionality.