Testing browser-initiated navigation.

[mikewest]

https://github.com/mikewest/sec-metadata/issues/20 aims to specify a feature which would distinguish between browser-initiated navigation (e.g. a user clicks on a bookmark, or types into the address bar). https://mikewest.github.io/sec-metadata/#directly-user-initiated spells out the rationale. This seems difficult to test.

For example:

• https://github.com/web-platform-tests/wpt/pull/15806 is a test which I expect will behave differently in different test harness implementations, as different harnesses trigger the initial navigation to a test in different ways.

• There's (quite intentionally!) no web-facing mechanism for pretending that a user typed in the address bar, or clicked a bookmark.

Perhaps it would be worthwhile to add new webdriver APIs?

/cc @johnwilander, with whom I discussed this on the last WebAppSec call.

[foolip]

@LukeZielinski, what do you think about extending https://w3c.github.io/webdriver/#navigate-to to support this? Assuming WebDriver could do this, how could you write tests using it? (The challenge is that you can't navigate away from the test itself, and if you window.open that window isn't entirely like a normal window.)

[mikewest]

I think that something like the following could work:

The harness navigates to test.html in whatever way makes sense. test.html calls window.open() to a puppet page, and drives it via postMessage() commands. That page can use the webdriver APIs to navigate itself in various ways, and postMessage() content back to test.html.

That needs some thought, though, as I worry that that will break down as window.opener is cleared by some UAs in some scenarios (cross-origin navigation, for example). If the windows are same-origin, they can BroadcastChannel to each other. If not, we may need to do more esoteric things like relying on communication through the server (using stashed values, etc)?

It's non-trivial. :)

[foolip]

If nothing else, the test page could poll the window by sending messages to it. That would not be elegant, of course.

@LukeZielinski this is a case of the type I've mentioned where it seems like the ability to write tests from the outside of the browser might be useful. Could something derivative of the WebDriver tests work? @KyleJu may also have ideas.

[LukeZielinski]

This is similar to what Mike suggested about using window.open(), but there is a webdriver command to open a new window. So could we have test.html open a new window, switch to it, and then navigate to the puppet page (all via webdriver)?

Would that perhaps avoid some of the issues with window.opener?

[mikewest]

So could we have test.html open a new window, switch to it, and then navigate to the puppet page (all via webdriver)?

Would that perhaps avoid some of the issues with window.opener?

There needs to be a communication channel between the opened window, and the window that did the opening. As noted above, we could route messages through the server (perhaps by delivering a token to the newly-opened window in the URL as a query parameter or fragment?), or we could BroadcastChannel for same-origin windows. Either of those paths seem viable, if somewhat complicated.

That said, it looks like there's also a mechanism to grab window handles (https://w3c.github.io/webdriver/#get-window-handles). Would that make it possible to postMessage() directly to the newly-opened window? That would be a lot simpler.

[LukeZielinski]

Does the communication channel need to go both ways, or just from parent to child?

The new-window webdriver command does return the handle of the newly-opened window, so could probably use that directly. It seems possible to extend navigate-to to take a window handle as a param as well (so you don't need to switch windows, although that seems minor), and then have a new webdriver endpoint to do the postMessage as you suggest. @foolip Does this approach sound reasonable to you?

[mikewest]

Does the communication channel need to go both ways, or just from parent to child?

We need to tell the page we're opening what to do (which, as noted above, could just be encoded in the URL in some way), and we need to get information back to the test so that we can verify the results.

The scheme you're laying out seems like it would be reasonable, but just to call out one bit that's glossed over: we need to define the way in which the webdriver API opens the window so that we can distinguish "The user typed this into the address bar and hit 'Go!'" from "The user clicked on a link that called window.open()." I think the intent in this conversation is for the webdriver API to only represent the former (which seems fine)?

https://mikewest.github.io/sec-metadata/#directly-user-initiated lays out a few other edge cases that it would be nice to test. For example, clicking on a link vs right-clicking on the link and selecting "open in new tab" from the context menu, or clicking on the browser's "back" button vs window.history.back(). It would be nice to have shared mechanisms for testing each.

[foolip]

@LukeZielinski returning window handled back to the test and letting the test then issue other WebDriver commands with those handles sounds like it would work, anything that's possible to do with WebDriver should ultimately be possible with this approach.

@mikewest for testing all those cases, is it WebDriver commands to open new windows in that way that are missing? Would it suffice to add enough modes to https://w3c.github.io/webdriver/#new-window, or does the opening have to be initiated from the right window to really work and have the right associations? In other words, is a "open as if from context menu for this element" comment necessary?

[anforowicz]

Would it suffice to add enough modes to https://w3c.github.io/webdriver/#new-window, or does the opening have to be initiated from the right window to really work and have the right associations? In other words, is a "open as if from context menu for this element" comment necessary?

Yes, the "open as if from context menu for this element" is important. See https://crbug.com/925322#c10 and https://crbug.com/925322#c11 for some context (currently internal, but should become public eventually + I've added you to CC so you should have access to the bug). I believe this bug highlights the importance of opening a new window by interacting with a specific element/link and via context menu (as opposed to, say, ctrl-clicking) on the page is important and distinct from just opening a new window by other means.

[LukeZielinski]

@anforowicz Would you be able to cc me on that bug as well? Apologies if any of the following is covered in there.

Just to clarify - are we concerned with how windows are opened, or just how they are navigated (or both)? My reading of mikewest/sec-metadata#20 is that the Sec-Fetch-Site header is sent when some already-open window is navigated. Is that right? If so, then I think we're talking about adding a parameter to https://w3c.github.io/webdriver/#navigate-to that specifies what the value of this header should be.

This also seems to map to the CDP command for navigation, where you can specify a TransitionType. Do those values cover the types of scenarios we care about here?

As for communicating between pages I wonder about the possibility of a WebDriver endpoint that would map a window handle to a window object, and then the test pages could window.postMessage directly to each other?

But in general, it sounds like adding testing support for this is feasible. Would you like to try to get this prototyped? Is there a simple example of a test we could work with?

[anforowicz]

@anforowicz Would you be able to cc me on that bug as well?

Done.

Just to clarify - are we concerned with how windows are opened, or just how they are navigated (or both)?

Both - these things are quite interconnected. We want to know how a navigation was initiated (through the trusted browser user interface VS through web content - e.g. through links) and this probably means that we want to know how windows are opened (via ctrl-clicking VS right-click-context-menu-open-in-new-tab VS right-click-in-tab-strip-duplicate).

My reading of mikewest/sec-metadata#20 is that the Sec-Fetch-Site header is sent when some already-open window is navigated. Is that right?

Not really. Sec-Fetch-Site should ideally be sent for all requests done by a browser, including navigations in a window opened via right-click-a-link-then-context-menu-then-open-in-new-tab.

This also seems to map to the CDP command for navigation, where you can specify a TransitionType. Do those values cover the types of scenarios we care about here?

No - in particular, a link value doesn't distinguish between clicking-a-link VS right-click-a-link-then-context-menu-then-open-in-new-tab.

As for communicating between pages I wonder about the possibility of a WebDriver endpoint that would map a window handle to a window object, and then the test pages could window.postMessage directly to each other?

I think this should work.

But in general, it sounds like adding testing support for this is feasible. Would you like to try to get this prototyped?

I think so.

Is there a simple example of a test we could work with?

Maybe a new test could be added under wpt/fetch/sec-metadata? Something like:

1. Open a web page with a link
2. right-click-a-link-then-context-menu-then-open-in-new-tab
3. Verify that the navigation request had Sec-Fetch-Site: none (I think this is the value we want, but whatever the desirable value is, having test coverage here would be great)

[guest271314]

@mikewest

I think that something like the following could work:

The harness navigates to test.html in whatever way makes sense. test.html calls window.open() to a puppet page, and drives it via postMessage() commands. That page can use the webdriver APIs to navigate itself in various ways, and postMessage() content back to test.html.

Have no experience using webdriver. Though this pattern using SharedWorker https://stackoverflow.com/a/38939251

window.worker = void 0, window.so = void 0;

fetch("https://path/to/shared_worker_script.js")
  .then(response => response.blob())
  .then(script => {
    console.log(script);
    var url = URL.createObjectURL(script);
    window.worker = new SharedWorker(url);
    console.log(worker);
    worker.port.addEventListener("message", (e) => console.log(e.data));
    worker.port.start();

    window.so = window.open("https://path/to/html_document_with_same_origin", "_blank");

    so.addEventListener("load", () => {
      so.worker = worker;
      so.console.log(so.worker);
      so.worker.port.addEventListener("message", (e) => so.console.log(e.data));
      so.worker.port.start();
      so.worker.port.postMessage("hi from " + so.location.href);
    });

    so.addEventListener("load", () => {
      worker.port.postMessage("hello from " + location.href)
    })

  });

might be able to be adjusted to suit the requirement?