Clarify navigation automation

[sadym-chromium]

Currently, from the WebDriver or WebDriver BiDi client's perspective, the prerendered traversable is not distinguishable from the traversable created by user. This can cause an observable difference in test behavior with prerendering on and off.

Scenario

• User communicates to the browser via automation protocol (let's say WebDriver BiDi).
• User creates _traversable1.
• User navigates _traversable1 to a _page1 with a link to a _page2 and speculation rules for prerendering the _page2.
• (if prerendering is enabled) New traversable _traversable2 is created and navigated to _page2.
• User clicks on link.
• (if prerendering is enabled) _traversable1 is destroyed, _traversable2 is activated.
• (if prerendering is disabled) _traversable1 is navigating to _page2.

Currently, there is no way for automation clients to detect the switch of the traversable when prerendering is enabled. This will lead to unexpected test results, when user navigates to a prerendered page and assumes the traversable has not changed.

[mathiasbynens]

Ref. https://github.com/w3c/webdriver-bidi/issues/321

[domenic]

Thanks for filing this issue! Can you state what the actual problem is, though? This all sounds like it's working more or less as I would expect. It's not expected that navigations without prerendering behave the same as navigations with prerendering; indeed, we definitely want them to behave differently, so that web developers can write tests which check that prerendering is working!

Currently, from the WebDriver or WebDriver BiDi client's perspective, the prerendered traversable is not distinguishable from the traversable created by user.

I'm not sure I understand this point. They are distinguishable from JavaScript, e.g. via document.prerendering. And my understanding was that WebDriver users could can JavaScript. So I think they should be distinguishable.

Is the idea that it would be helpful to make them easier to distinguish, without running JavaScript? That seems vaguely like a good idea to me.

Currently, there is no way for automation clients to detect the switch of the traversable when prerendering

Similarly, this is also detectable using JavaScript (e.g. with the prerenderingchange event).

[sadym-chromium]

Thanks for filing this issue! Can you state what the actual problem is, though? This all sounds like it's working more or less as I would expect. It's not expected that navigations without prerendering behave the same as navigations with prerendering; indeed, we definitely want them to behave differently, so that web developers can write tests which check that prerendering is working!

This is a problem for test developers who are not aware of prerendering. For them enabling prerendering will be a breaking and unexpected change, as it will cause switching traversables without any visual indication of that. The developer can be confused by that, and would require deep debugging in order to figure out that it's prerendering caused this.

Is the idea that it would be helpful to make them easier to distinguish, without running JavaScript? That seems vaguely like a good idea to me.

The idea is to add an event or other mechanism, signaling that the prerendering happened, or that the switch has happened. This would allow testing frameworks communicate what is happened to the test developers.

[domenic]

This is a problem for test developers who are not aware of prerendering. For them enabling prerendering will be a breaking and unexpected change, as it will cause switching traversables without any visual indication of that.

I'm unclear what scenario you're envisioning here. In what scenario is a developer adding prerendering to their site, and not aware that they're doing so?

The idea is to add an event or other mechanism, signaling that the prerendering happened, or that the switch has happened.

As I mentioned, these events already exist. Can you explain what's unsatisfactory about the existing ones?

[sadym-chromium]

I'm unclear what scenario you're envisioning here. In what scenario is a developer adding prerendering to their site, and not aware that they're doing so?

As I understand, the assumption is that testers should always be aware if a site employs prerendering.

However, this assumption may not always be true. E.g. in the following cases:

• Consider a large company with a custom application server (like a modified Tomcat with added services and optimizations) used across different teams. Each team independently tests its services, perhaps using Puppeteer. If prerendering is enabled for certain pages (e.g., the login page), these teams might encounter failing tests without clear reasons or solutions.
• Another scenario is when the tests are written after the site was developed by other people/team/contracters.

Testing frameworks (like Puppeteer or Selenium) could potentially address this by subscribing to relevant events and providing informative messages during runtime to aid debugging.

As I mentioned, these events already exist. Can you explain what's unsatisfactory about the existing ones?

This relates to my proposal to extend BiDi with prerendering information. The prerenderingchange event occurs within the page's JavaScript context. For BiDi testing frameworks (like Puppeteer or Selenium) to utilize this, a script must be evaluated on each page to subscribe to this event and communicate changes to the testing framework. However, this necessitates evaluating a script on every document, which is not ideal.

A possible solution is to extend the WebDriver BiDi browsingContext.Info type with an optional ? "goog:prerendering": true parameter. This would allow testing frameworks to inform users that a browsing context is prerendered by simply checking this flag. This extension could be incorporated into the prerendering specification instead of the main WebDriver BiDi spec, similar to how the WebBluetooth specification handles automated testing. This would involve phrase like "Extend get the navigable info algorithm with ...", which should require only a minor modification.

Implementing this minor change could reduce debugging time, increase awareness of prerendering, and enhance the overall testing experience.

Please let me know if you find this proposal valuable, and I can provide more details on the implementation. Feel free to close the issue if you believe the problem described is too minor to warrant this change.

[domenic]

Please let me know if you find this proposal valuable, and I can provide more details on the implementation. Feel free to close the issue if you believe the problem described is too minor to warrant this change.

I don't have a strong opinion on whether or not this is valuable. Do we have any testimony from web developers working with prerendering that are in the sort of situation you outline? My default for normal APIs would be to wait until we hear from such developers, but I don't have experience with how testing APIs are developed. Maybe they are usually developed in anticipation of web developer needs, instead of in reaction to them. So I'd rather leave that determination up to you.

If you think this is valuable, and furthermore that it's valuable to get this information without running JavaScript in the context of the page, then spec updates to provide this information would be welcome as pull requests to this repository (or to the WebDriver BiDi spec, as appropriate). I don't know why you added a goog: prefix though; does that mean "Google"? That doesn't seem appropriate for a web specification.

I'm also curious on your take on https://github.com/w3c/webdriver-bidi/issues/321 .

[sadym-chromium]

I don't know why you added a goog: prefix though.

Oh, right, it should not have the goog: prefix, just prerendering.

I'm also curious on your take on w3c/webdriver-bidi#321 .

I guess that issue is a duplicate of this one, or this one a dup of that.