w3c / webdriver

Remote control interface that enables introspection and control of user agents.
https://w3c.github.io/webdriver/
Other
681 stars 195 forks source link

Unknown capabilities cause session creation to fail during `firstMatch` validation #1528

Open foolip opened 4 years ago

foolip commented 4 years ago

In https://github.com/w3c/webdriver-bidi/pull/24/files#r434807040 I did some testing of "firstMatch" handling. Simplifying it here, I tried POSTing this payload to /session with chromedriver, geckodriver and safaridriver:

{
  "capabilities": {
    "firstMatch": [
      {
        "futureThing": 42
      },
      {}
    ]
  }
}

Tested with curl -d '{"capabilities":{"firstMatch":[{"futureThing": 42},{}]}}' "http://localhost:4444/session".

All 3 drivers fail with "invalid argument" error with these messages:

Per spec I think this is not intended, as a "validated capabilities" list is constructed.

I want to check in with those more familiar with the spec before doing anything more:

jgraham commented 4 years ago

So apparently I entirely forgot where the spec ended up on this.

At the moment, if we don't recognise a capability at all we fail validation, and failing validation ends up returning an error from the whole processing capabilites algorithm. So implementations are correct per spec, and a client that wanted to use some non-extension capabilities that may or may not be recognised would have to make multiple attempts at constructing a session.

The bidi stuff seems like an example of where this is bad, but I don't recall all the arguments that led to the current setup. Maybe @shs96c does.