w3c / webdriver-bidi

Bidirectional WebDriver protocol for browser automation
https://w3c.github.io/webdriver-bidi/
378 stars 42 forks source link

Network events and interception for data: scheme #727

Open juliandescottes opened 5 months ago

juliandescottes commented 5 months ago

At the moment, Chrome supports network events for data: URIs but not interception, and Firefox doesn't support it at all but implementation is in progress at https://bugzilla.mozilla.org/show_bug.cgi?id=1805176

From a spec point of view, our PR for the fetch spec bypasses some BiDi events in case of a data: URI, because at step 12 of Main Fetch, we will only run scheme fetch and we never go into "HTTP fetch" or "HTTP-network-or-cache fetch". This is where we normally trigger beforeRequestSent and responseStarted, so we won't get those events for a data: URI.

However we would still emit responseCompleted at step 3.3.9 of fetch response handover (or a fetchError...).

This is convenient because we bypass the phases where we would be able to intercept the request (which probably doesn't make sense for data URI), but are we OK with only emitting responseCompleted for data: URI network events?

cc @jgraham

@OrKoN Do you know which network events Chrome emits for a data: URI?

juliandescottes commented 5 months ago

Considering we have the following assert in responseCompleted

Assert: before request sent map[request] contains redirect count.

So even if we are fine with only emitting one event for data URIs, we will still need to update the BiDi spec to support that use case?

OrKoN commented 5 months ago

Chrome emits Network.requestWillBeSent and Network.responseReceived which would match network.beforeRequestSent and network.ResponseStarted/network.responseCompleted without ability to intercept.

css-meeting-bot commented 5 months ago

The Browser Testing and Tools Working Group just discussed Network events for data URIs, and agreed to the following:

The full IRC log of that discussion <jgraham> Topic: Network events for data URIs
<jgraham> github: https://github.com/w3c/webdriver-bidi/issues/727
<orkon> jdescottes: Firefox is implementing data URIs for network events. So Firefox only emits responseCompleted. So BiDi spec expects a beforeRequest event before responseCompleted.
<orkon> jdescottes: that is also related to the topic of network interception what we should for data URIs and if we only emit response completed and that is fine since it cannot be intercepted at this stage
<orkon> jdescottes: do we want the full or limited set of events? do we want to have the interception for those requests?
<orkon> q+
<jgraham> ack next
<jgraham> ScribeNick: jgraham
<jgraham> orkon: Chrome emits those events but doesn't support interception for data requests. Data requests don't come from the network, so how much it's applicable to data URLs is a question. It would be great to have the full set of events from the client perspective so you get the same workflow as for other requests. At the same time if there's one event with all the data then it's probably acceptable.
<jgraham> q+
<orkon> ack next
<orkon> ScribeNick: orkon
<orkon> jgraham: we could a completely separate events for data URIs and if that just has a single event it might break a few invariants
<jdescottes> q+
<jgraham> s/break a few/break fewer/
<jgraham> ack next
<orkon> jdescottes: so a separate was a good solution. So we will still need to modify the spec that data URIs do not trigger responseCompleted. In DevTools UI data URIs usually show them in the same place. So users are likely to treat data URI as regular URIs. So I would prefer to have the full set of events as well.
<orkon> jgraham: sending one event instead of three would be more efficient
<orkon> q+
<jgraham> ack next
<jgraham> ScribeNick: jgraham
<jgraham> orkon: I'm wondering if there's a use case for data urls. Potentially an app could fetch some data and insert it as data uris, and maybe there's some scenario where you want to modify that. But it's an argument for having three events because that would make it easier to cover data uris for interception vs a single event.
<orkon> ScribeNick: orkon
<orkon> jgraham: it sounds like people are in general in favor of making 3 events
<orkon> jgraham: it sounds like we should change the spec change to make sure we are emitting them
<simonstewart> Cheerio, everyone
<jgraham> RRSAgent: make minutes
<RRSAgent> I have made the request to generate https://www.w3.org/2024/06/12-webdriver-minutes.html jgraham
<jgraham> zakim, bye
<Zakim> leaving. As of this point the attendees have been orkon, jimevans, sasha, jgraham, jdescottes, lightning00blade, simonstewart
<jgraham> RRSAgent: bye
<RRSAgent> I see 1 open action item saved in https://www.w3.org/2024/06/12-webdriver-actions.rdf :
<RRSAgent> ACTION: item to Aex to extract the relevant pieces from PR 565 and create a new PR [1]
<RRSAgent> recorded in https://www.w3.org/2024/06/12-webdriver-irc#T16-22-06