There seems to be a cache somewhere between the server hosting the tests (which runs this plugin) and the browsers we're testing on Sauce Labs. Without this header, this cache serves responses generated for one browser's requests to those of another browser, which may have different compliation settings.
These different browsers which seem to be having their responses mixed up are running in different OSs on different VM instances, so I don't think this cache could possibly be running anywhere on the client VM. Also, I've checked that the requests are in fact reaching the server and responses are being compiled - the response is just different by the time the browser receives it. This makes me think that the culprit is Sauce Labs Proxy, but the docs for Sauce Labs Proxy claim that it no longer caches.^1 They also recently deprecated a --no-proxy-caching flag^2, so I'm skeptical.
Setting Cache-Control: no-cache can't be used to work around this because @web/test-runner unconditionally installs server middleware[^3] that almost always sets Cache-Control: public, max-age=31536000 in non-interactive scenarios.^4
Fortunately, this unknown cache seems to be responsive to the Vary request header. The plugin specifically uses the User-Agent request header to determine compilation settings, so adding Vary: User-Agent will cause the cache to use fresh responses for any change in the UA.
There seems to be a cache somewhere between the server hosting the tests (which runs this plugin) and the browsers we're testing on Sauce Labs. Without this header, this cache serves responses generated for one browser's requests to those of another browser, which may have different compliation settings.
These different browsers which seem to be having their responses mixed up are running in different OSs on different VM instances, so I don't think this cache could possibly be running anywhere on the client VM. Also, I've checked that the requests are in fact reaching the server and responses are being compiled - the response is just different by the time the browser receives it. This makes me think that the culprit is Sauce Labs Proxy, but the docs for Sauce Labs Proxy claim that it no longer caches.^1 They also recently deprecated a
--no-proxy-cachingflag^2, so I'm skeptical.Setting
Cache-Control: no-cachecan't be used to work around this because@web/test-runnerunconditionally installs server middleware[^3] that almost always setsCache-Control: public, max-age=31536000in non-interactive scenarios.^4[^3]:
cacheMiddlewareis added before...config.middleware, giving it higher precedence. It also callsawait ctx.next()before doing any work, so it always runs last and overwrites anyCache-Controlheader added by any user middleware: https://github.com/modernweb-dev/web/blob/da7901116322fc112ea45413e6be87f3f770f15e/packages/test-runner-core/src/server/TestRunnerServer.ts#L50Fortunately, this unknown cache seems to be responsive to the
Varyrequest header. The plugin specifically uses theUser-Agentrequest header to determine compilation settings, so addingVary: User-Agentwill cause the cache to use fresh responses for any change in the UA.