search

web-platform-tests / rfcs

web-platform-tests RFCs
74 stars 63 forks source link

RFC 185: Add WebDriver BiDi support to testdriver.js #185

Closed sadym-chromium closed 2 months ago

sadym-chromium commented 3 months ago

Re-apply https://github.com/web-platform-tests/rfcs/pull/182

Add “testdriver.js” support for WebDriver BiDi events and actions.

Preview

jgraham commented 3 months ago

As discussed in the infra meeting I'm going to review this, but it may take some time since it's a rather substantial proposal that is likely to have significant knock-on effects. It would also be useful to get review from anyone who is not planning to use the webdriver backend to wptrunner in the near future; that might include WebKit (e.g. @gsnedders) and Servo (@mrobinson perhaps).

OrKoN commented 3 months ago

Friendly ping to review the RFC @jgraham @gsnedders. There is a meeting tonight that offers an opportunity to discuss the proposed changes if there are any questions.

jgraham commented 3 months ago

Just FYI, I have started looking at this. I'm not sure if it's making the right set of tradeoffs in the current form, but I need more time to look again.

sadym-chromium commented 2 months ago

I'd also like to see examples of how we could use the API for use cases that currently aren't possible or are prohibitively difficult, for example running actions in cross-origin windows. BiDi should unlock that, since it adds the ability to message arbitrary contexts. However the focus on just getting console.log events doesn't really reveal if this design is a useful stepping stone to that outcome. I worry that choices here could impede our ability to make the best use of BiDi in the future, if tests end up relying on accidental features of the design.

The core issue with handling several browsing context from a testdriver is in the fact the testdriver is not aware of browsing context ids. I see two approaches to address this:

  1. When BiDi is available, allow testdriver to use window itself instead of the window.__wptrunner_id as a context. On the testdriver side, parse the testdriver messages and extract the browsing context ID.
  2. Make testdriver be aware of it's own browsing context ID. This can be done by one of this ways:
    1. Add to script.addPreloadScript a new argument type browsingContext, which will be resolved to the actual browsingContext ID when called.
    2. Set a promise window.__browsing_context_id via script.addPreloadScript, and resolve it via script.evaluate.
sadym-chromium commented 2 months ago

I'd also like to see examples of how we could use the API for use cases that currently aren't possible or are prohibitively difficult, for example running actions in cross-origin windows. BiDi should unlock that, since it adds the ability to message arbitrary contexts. However the focus on just getting console.log events doesn't really reveal if this design is a useful stepping stone to that outcome. I worry that choices here could impede our ability to make the best use of BiDi in the future, if tests end up relying on accidental features of the design.

I updated the example by adding a parameter contexts to subscribe action, which can be either a window proxy or a string representing browsing context id. This became possible because of the _get_next_message uses BiDi when it is available.

sadym-chromium commented 2 months ago

@jgraham do you think we need to extend other test in the example to make a more explicit use case of BiDi?

sadym-chromium commented 2 months ago

I'd also like to see examples of how we could use the API for use cases that currently aren't possible or are prohibitively difficult, for example running actions in cross-origin windows. BiDi should unlock that, since it adds the ability to message arbitrary contexts. However the focus on just getting console.log events doesn't really reveal if this design is a useful stepping stone to that outcome. I worry that choices here could impede our ability to make the best use of BiDi in the future, if tests end up relying on accidental features of the design.

Here is a draft example of what it can look like: https://github.com/sadym-chromium/wpt/pull/450. @jgraham @gsnedders WDYT?

foolip commented 2 months ago

I'll go ahead and merge this now. The RFC has been open for over a month and it's fair to say that we've ensured "that all participants have time to consider it." If anyone from Apple wants to leave feedback on the design, it's not too late for that even after implementation has begun.