w3c / webdriver-bidi

Bidirectional WebDriver protocol for browser automation
https://w3c.github.io/webdriver-bidi/
356 stars 39 forks source link

Add `script.callFunction` command #140

Closed sadym-chromium closed 2 years ago

sadym-chromium commented 2 years ago

As a BiDi user, I want to be able to invoke some custom function with some custom arguments on the page context. For example, I want to be able to invoke (text) => {this.innerHtml=text} with custom this and text.

It can be done:

  1. As a part of the script.evaluate command by adding optional parameters thisObject and argument. It can be quite ugly, as expression changes meaning from expression to be evaluated and returned as a result to expression to return a function which will be invoked with the given params and each result will be returned to user.
  2. As another command script.invoke with an expression string returning function to be invoked, thisObject and argument.

I'm inclined to go with the second way.

There is another open question:

  1. How should the arguments be presented? As a user, I want to be ale to use both some constants, simple objects like {a: 1}, and RemoteObject return by e.g. script.evaluate.

For primitive type it is logical to use the remote value serialisation. But for non-primitive it's enough to have only objectId, even without type. Should we send {type: 'object', objectId: 'SOME_OBJECT_ID'}, or we don'r require type?

For non-primitive objects I want to use as an argument, if I want to pass some new object as an argument, which was not presented in BiDi yet (like {a: 1}), I want to use be able to use BiDi serialisation protocol. But it requires making an objectId optional.

UPD:

As discussed, command script.callFunction would be more readable.

jgraham commented 2 years ago

Yes, I was assuming we'd have something like script.call[FunctionOn].

For arguments, we could indeed allow a serialization like {objectId: <object id string>} to refer to an object on the remote end. The question is whether to allow type and value fields. The advantage of that would be allowing the exact same object from a response to be used in an argument. The disadvantage is that it's unclear what to do if there's a mismatch (e.g. if you pass in type: "object" but once you lookup the object id get a map).

I think I favour minimisng the amount of validation having a schema like:

Argument = (
   RemoteValueArgument //
   ValueArgument
)

RemoteValueArgument = {
    objectId: text
}

ValueArgument = {
    type: text,
    value: Argument
}
css-meeting-bot commented 2 years ago

The Browser Testing and Tools Working Group just discussed Sending data from the local end (client) to the remote end (browser).

The full IRC log of that discussion <AutomatedTester> topic: Sending data from the local end (client) to the remote end (browser)
<AutomatedTester> github: https://github.com/w3c/webdriver-bidi/issues/140
<AutomatedTester> jgraham: how do we serialise the arguments from the client to the browser
<AutomatedTester> ... and this came up from a PR that sadym did
<AutomatedTester> ... and we don't want to copy selenium
<AutomatedTester> ... so if its an ID we will run it from what's stored
<AutomatedTester> ... so do people have opinions on what is sent to the browser?
<AutomatedTester> ... [describes the events and seriablisable args]
<AutomatedTester> ... so we can either have the client have a look process or have a symetrical serialisation process
<sadym> q+
<AutomatedTester> ack sadym
<AutomatedTester> sadym: So my idea was to have 1 source of truth
<AutomatedTester> ... and we mirror from the browser to the client
<brwalder> q+
<AutomatedTester> ... e.g. If I create an object on the client side and passed it to the browser what would happen
<AutomatedTester> jgraham: so on the browser side it would look like a immutable platform object
<AutomatedTester> ... we have some strong use cases for sending back full objects to the client and not that much in the other directions
<AutomatedTester> q?
<AutomatedTester> sadym: the parametere in the `invoke` do we need this object reference? In CDP there is an implementation but it seems to be syntatic sugar
<AutomatedTester> jgraham: I agree with you for the initial implementation
<AutomatedTester> ... although we should look and see what puppeteer/playwright do
<AutomatedTester> q?
<AutomatedTester> ack brwalder
<AutomatedTester> brwalder: should we maintain object ideas on the client side
<AutomatedTester> ... I agree with sadym here that the browser should be the source of truth
<AutomatedTester> ... if we don't do that we have a lot of complexity on clients
<AutomatedTester> ... [describes a scenario]
<AutomatedTester> ... and I am not convinced the added complexity in browsers is worth it
<AutomatedTester> q?
<sadym> q+
<AutomatedTester> jgraham: I don't think there is a lot of enthusiam for supporting this now
<AutomatedTester> ... so we'll do the simple thing which is great
<AutomatedTester> ack sadym
<AutomatedTester> sadym: one other scenario when a client receives an object with args and then sends it back the browser the args could be ignored
<AutomatedTester> ... and there wouldnt be a collision with things going back and forth
<AutomatedTester> q?
sadym-chromium commented 2 years ago

Merged in #142