w3c / webdriver-bidi

Bidirectional WebDriver protocol for browser automation
https://w3c.github.io/webdriver-bidi/
351 stars 38 forks source link

Add `browsingContext` parameter to Network Interception #515

Closed thiagowfx closed 6 months ago

thiagowfx commented 1 year ago

FR: This is pretty much the preload script equivalent for Network Interception (c.f. https://github.com/w3c/webdriver-bidi/issues/373).

Currently, Network Interception is done globally, to all browsing contexts. There is no way for interceptions to be added to specific browsing contexts, and there is no way for intecepts to determine in which browsing context it is applied to.

This use case has been requested by the Puppeteer team.

We could look into adding an optional context: browsingContext.BrowsingContext parameter to e.g. network.AddIntercept.

thiagowfx commented 1 year ago

cc @mathiasbynens @OrKoN @Lightning00Blade FYI.

juliandescottes commented 1 year ago

@thiagowfx the intercepts only run for the events which have an active subscription, so if you subscribe to network events for a specific context, the intercepts will already be limited to those.

I imagine that puppeteer subscribes globally, and that's what makes this inconvenient? On the other hand having to specify contexts both for the network events AND for the intercepts feels odd.

OrKoN commented 1 year ago

Puppeteer subscribes to network events in all contexts for other use cases (such as navigation), but the user defines which contexts need to have interception enabled additionally (via page.setRequestInterception). As there could be quite large overhead connected with the interception, I think it makes sense to allow scoping intercepts to particular contexts?

juliandescottes commented 1 year ago

re: overhead, if you're already watching for network events, it should only be a matter of matching the URL of the event against a URL pattern.

But based on the APIs you described, it seems mandatory to add this feature in order to support puppeteer anyway.

In practice it means the intercepts will only be valid for the intersection of contexts specified for the network events and for the intercept. But hopefully in most cases either users will only specify a context for either the network events, or for the intercepts, and not for both.

OrKoN commented 1 year ago

re: overhead, if you're already watching for network events, it should only be a matter of matching the URL of the event against a URL pattern.

I meant that there is an overhead related to the round trip between the client and the server to unblock each of the requests.

juliandescottes commented 1 year ago

I meant that there is an overhead related to the round trip between the client and the server to unblock each of the requests.

Oh ok, I had in mind that the URL patterns would help to avoid getting unrelated requests intercepted. But indeed, if there's another context which makes a request matching a pattern, there will be significant overhead.

OrKoN commented 1 year ago

Puppeteer intercepts everything for a page and the user provides a callback where they can decide what they do with each request. E.g., you can block every 3rd request for the same URL etc.