web-platform-tests / wpt

Test suites for Web platform specs — including WHATWG, W3C, and others
https://web-platform-tests.org/
Other
4.98k stars 3.09k forks source link

Handle any.js or variants in a window global scope which is controlled by a service worker #11962

Open makotoshimazu opened 6 years ago

makotoshimazu commented 6 years ago

We run *.any.js on window, dedicatedworker, serviceworker, sharedworker scopes, but we don't have a way to specify those global scopes hosted by a service worker. It changes the behavior of network requests, so it will exercise more codes.

For example, wpt/fetch/api are written by *.any.js but all network requests in those tests won't be routed to service workers.

foolip commented 6 years ago

Hmm, interesting. Could the same service worker be used for all such tests, just to see that it is "transparent"?

@annevk, do you think this would be technically straightforward to implement?

makotoshimazu commented 6 years ago

I think service workers should be transparent to those tests, but service workers themselves may be needed to be modified in some ways independently with tests (like caching, fetching explicitly from the network in the service worker, and just returning with nothing else).

foolip commented 6 years ago

Right, so the point of this is to discover bugs in the service worker implementation, cases where the behavior has changed even though the service worker isn't set up to do anything?

Are any tests like this written manually right now?

annevk commented 6 years ago

How does this make sense for the serviceworker "scope" of an any.js test? Adding more "scopes" is somewhat straightforward, but it would be good to make the proposal a bit more concrete.

foolip commented 6 years ago

@makotoshimazu can you point to some tests that do this kind of thing manually, presumably duplicating some code, to illustrate the problem?

makotoshimazu commented 6 years ago

Thanks for digging into this!

Scope seems depending on the performance of the test. Tests should work regardless of its scopes if we only registers "transparent" service workers which I mentioned in the bottom. More registrations would improve concurrency of test execution.

As an exmample of manual registration, fetch-event.https.html is a test that adds a scope and loads a resource, but I don't think this test is a representative of that kind of thing since it's only for pages controlled by a service worker (of course some of tests whose service worker falls back to the network should work even when service worker doesn't control the page, though). And during looking for a good example to put here, I realized that unfortunately we don't have a good coverage of tests for pages controlled by a service worker unfortunately...

I think those four cases are popular and should be transparent to tests. All any.js tests probably should pass with those service workers.

annevk commented 4 years ago

Sorry for not following up here. It seems like a reasonable suggestion to increase test coverage for service workers, but should probably be opt-in per test to avoid clashes?

annevk commented 4 years ago

Note also #21163 by the way. Currently many tests could be run in a service worker, but don't.