microsoft / vscode-test-web

Node module to help testing VS Code web extensions.
MIT License
60 stars 23 forks source link

CDN requests cause occasional failures in Github Workflow #106

Closed minestarks closed 1 year ago

minestarks commented 1 year ago

Would it be possible for @vscode/test-web to serve all files from localhost instead of fetching some from the web when testing?

Our VS Code extension tests keep hitting 504 Gateway errors when making requests to the VS Code CDN.

Please see https://github.com/microsoft/qsharp/actions/runs/6513910585 for a sample run showing these 504s. I added some instrumentation here to log all http requests made by VS Code, to try and root cause. You can see how the run times out shortly after one of these requests fail:

logs ``` response: 504 https://v--1k067d9ucs6cb62mmh7vhhdgleuqjvoc0mefdighcels0cm0akel.vscode-cdn.net/stable/f1b07bd25dfad64b0167beb15359ae573aecd2cc/out/vs/workbench/api/worker/extensionHostWorker.js Loading "vs/workbench/api/worker/extensionHostWorker" failed Error Here are the modules that depend on it: JSHandle@object response: 504 https://0268f9e0hfrgqbbpg0d1963mkls2vpthd5fp84a2ja6pof3cjn7p.vscode-cdn.net/stable/f1b07bd25dfad64b0167beb15359ae573aecd2cc/out/vs/workbench/contrib/webview/browser/pre/fake.html?id=27d92fc5-c750-4d58-a5a5-99a29ef71cf4 [JavaScript Warning: "This page is in Quirks Mode. Page layout may be impacted. For Standards Mode use “”." {file: "https://0268f9e0hfrgqbbpg0d1963mkls2vpthd5fp84a2ja6pof3cjn7p.vscode-cdn.net/stable/f1b07bd25dfad64b0167beb15359ae573aecd2cc/out/vs/workbench/contrib/webview/browser/pre/fake.html?id=27d92fc5-c750-4d58-a5a5-99a29ef71cf4" line: 0}] [JavaScript Warning: "The character encoding of a framed document was not declared. The document may appear different if viewed without the document framing it." {file: "https://0268f9e0hfrgqbbpg0d1963mkls2vpthd5fp84a2ja6pof3cjn7p.vscode-cdn.net/stable/f1b07bd25dfad64b0167beb15359ae573aecd2cc/out/vs/workbench/contrib/webview/browser/pre/fake.html?id=27d92fc5-c750-4d58-a5a5-99a29ef71cf4" line: 0}] req: GET http://localhost:3000/static/build/out/vs/workbench/contrib/welcomeGettingStarted/common/media/dark.png req: GET http://localhost:3000/static/build/out/vs/workbench/contrib/welcomeGettingStarted/common/media/light.png req: GET http://localhost:3000/static/build/out/vs/workbench/contrib/welcomeGettingStarted/common/media/dark-hc.png req: GET http://localhost:3000/static/build/out/vs/workbench/contrib/welcomeGettingStarted/common/media/light-hc.png GET /static/build/out/vs/workbench/contrib/welcomeGettingStarted/common/media/dark.png 200 0.917 ms - 10566 GET /static/build/out/vs/workbench/contrib/welcomeGettingStarted/common/media/light.png 200 0.542 ms - 10154 GET /static/build/out/vs/workbench/contrib/welcomeGettingStarted/common/media/dark-hc.png 200 2.273 ms - 10615 GET /static/build/out/vs/workbench/contrib/welcomeGettingStarted/common/media/light-hc.png 200 8.638 ms - 10615 response: 200 http://localhost:3000/static/build/out/vs/workbench/contrib/welcomeGettingStarted/common/media/dark.png response: 200 http://localhost:3000/static/build/out/vs/workbench/contrib/welcomeGettingStarted/common/media/light.png response: 200 http://localhost:3000/static/build/out/vs/workbench/contrib/welcomeGettingStarted/common/media/dark-hc.png response: 200 http://localhost:3000/static/build/out/vs/workbench/contrib/welcomeGettingStarted/common/media/light-hc.png The Web Worker Extension Host did not start in 60s, that might be a problem. ```

I investigated whether I could keep VS Code from hitting the CDN at all, since ideally CI tests would not be making web requests. However, it seems that even when a local VS Code build is used (and served from static/build), certain files are requested from the CDN.

workerMain.js, editorSimpleWorker.nls.js extensionHostWorker.js ...etc.

Here is some logging to demonstrate the issue in this repo's sample tests: https://github.com/microsoft/vscode-test-web/compare/main...minestarks:vscode-test-web:minestarks/request-log

yarn sample-tests
REQUEST: https://v--0avn07n8vsgclmgg5rjtqmgcjvdb805k53ucmiacu1m74g5mpgtb.vscode-cdn.net/insider/4678919d52909efc7c4ab958a2e5c8f45d50ffc3/out/vs/base/worker/workerMain.js
REQUEST: blob:http://localhost:3000/b8a82426-4519-4933-b5da-4a0d32892844
REQUEST: https://1kv163i31g272idmicbb2rfifejc5p7gofj9e65cb08v9h5rb6rt.vscode-cdn.net/insider/4678919d52909efc7c4ab958a2e5c8f45d50ffc3/out/vs/workbench/contrib/webview/browser/pre/fake.html?id=fa3be03e-4d47-403a-b0b8-606efef1617c
REQUEST: https://v--0avn07n8vsgclmgg5rjtqmgcjvdb805k53ucmiacu1m74g5mpgtb.vscode-cdn.net/insider/4678919d52909efc7c4ab958a2e5c8f45d50ffc3/out/vs/editor/common/services/editorSimpleWorker.nls.js
REQUEST: https://v--0avn07n8vsgclmgg5rjtqmgcjvdb805k53ucmiacu1m74g5mpgtb.vscode-cdn.net/insider/4678919d52909efc7c4ab958a2e5c8f45d50ffc3/out/vs/workbench/api/worker/extensionHostWorker.js
REQUEST: https://github.gallerycdn.vsassets.io/extensions/github/copilot/1.126.491/1697213545029/Microsoft.VisualStudio.Code.Manifest
REQUEST: https://v--0avn07n8vsgclmgg5rjtqmgcjvdb805k53ucmiacu1m74g5mpgtb.vscode-cdn.net/insider/4678919d52909efc7c4ab958a2e5c8f45d50ffc3/out/vs/workbench/api/worker/extensionHostWorker.nls.js

Any workaround suggestions appreciated.

minestarks commented 1 year ago

A proposed fix is at #107 .