Open d4h0 opened 1 year ago
I ran deno run --unstable --allow-all --inspect-brk main.ts
and examined the call stack.
The cause is that stdio[3]
and stdio[4]
are undefined here. Through these pipes Playwright communicates with the launched Chromium process.
const stdio = launchedProcess.stdio as unknown as [NodeJS.ReadableStream, NodeJS.WritableStream, NodeJS.WritableStream, NodeJS.WritableStream, NodeJS.ReadableStream];
transport = new PipeTransport(stdio[3], stdio[4]);
It seems deno's child_process
shim (and ultimately Deno.Command
) does not yet support to open fds other than std{in,out,err}
.
https://github.com/denoland/deno_std/blob/0.167.0/node/internal/child_process.ts#L153
http://docs.libuv.org/en/v1.x/guide/processes.html
This is correct. Deno doesn't currently support libuv's cross platform api for fds other than std{in,out,err}. via https://github.com/denoland/deno/issues/16298
libuv's icon is a dino with a unicorn. Kind of makes you wonder. :-)
But yes. My incident is about supporting all of libuv. Not just a few additional pipelines beyond stdin/stdout/stderr.
As @tamusjroyce wrote:
There are two ways for playwright and puppeteer to communicate with chrome/edge. websockets or signals & streams.
So if signals & streams are too difficult to implement (in the short-term), then Websockets might be a more feasible way to run Playwright under Deno.
I actually tried that in the past already (see here), but that attempt also failed.
After re-reading everything about Deno/Playwright, I realized that there might be different reasons for why the Websocket route doesn't work, so I tried again.
The resulting error is as follows:
error: Uncaught Error: browserType.connectOverCDP: WebSocket error: ws://127.0.0.1:39959/devtools/browser/dc49d18d-00b4-4a28-9de9-92208e57363d 101 Switching Protocols
=========================== logs ===========================
<ws connecting> ws://127.0.0.1:39959/devtools/browser/dc49d18d-00b4-4a28-9de9-92208e57363d
<ws unexpected response> ws://127.0.0.1:39959/devtools/browser/dc49d18d-00b4-4a28-9de9-92208e57363d 101 Switching Protocols
<ws error> error WebSocket was closed before the connection was established
<ws connect error> ws://127.0.0.1:39959/devtools/browser/dc49d18d-00b4-4a28-9de9-92208e57363d WebSocket was closed before the connection was established
<ws disconnected> ws://127.0.0.1:39959/devtools/browser/dc49d18d-00b4-4a28-9de9-92208e57363d code=1006 reason=
<ws error> error WebSocket was closed before the connection was established
<ws connect error> ws://127.0.0.1:39959/devtools/browser/dc49d18d-00b4-4a28-9de9-92208e57363d WebSocket was closed before the connection was established
<ws disconnected> ws://127.0.0.1:39959/devtools/browser/dc49d18d-00b4-4a28-9de9-92208e57363d code=1006 reason=
============================================================
If I run my test script (see below) with Node.js, everything works as expected.
So somehow Websockets are not behaving as expected under Deno.
If someone wants to give this a try, here is how to do it:
main.ts
with the following content:
import { chromium } from 'npm:playwright';
// import { chromium } from 'playwright';
// To run this script with Node.js (e.g. for testing),
// replace the above lines (don't forget to install
// Playwright via npm i playwright
)
async function main() { // If this line is uncommented, the script fails. // This will print the executed command, which we'll // need soon: const browser = await chromium.launch({ headless: false });
// If this line is uncommented (the above line should be commented out),
// then Playwright will connect to the browser via Websocket.
// $WEBSOCKET_ADDRESS
needs to be replaced with a valid Websocke
// address (see later step)
// const browser = await chromium.connectOverCDP("$WEBSOCKET_ADDRESS");
const page = await browser.newPage();
await page.goto('http://example.com');
const title = await page.title();
console.log(=> Page title: ${title}
)
await browser.close();
};
main()
2. Run the script via `deno run --unstable --allow-all main.ts`
4. Copy the command that was executed from the printed log (the part after `<launching>`)
3. Replace `--remote-debugging-pipe` with `--remote-debugging-port=0`
6. Execute the resulting command
7. This will start a browser, which will print a line starting with `DevTools listening on ws://`
8. Copy the displayed Websocket URL (the text after `DevTools listening on ` till the end of the line)
11. Replace `$WEBSOCKET_ADDRESS` in the script with the Websocket URL you just copied
10. Switch the script to use Websockets (see the comment above the `chromium.connectOverCDP` line)
12. Execute `deno run --unstable --allow-all main.ts`
As mentioned above, this will result in a Websocket error.
Does anyone have any clue what the problem could be?
I've set up a "man-in-the-middle" proxy to see what Websocket requests Node.js and Deno are sending, to figure out what's going on.
Here is the request and response when Node.js executes the above script:
Request:
GET /devtools/browser/dc49d18d-00b4-4a28-9de9-92208e57363d HTTP/1.1
Sec-WebSocket-Version: 13
Sec-WebSocket-Key: V9zFa5WtpjHUIIAfif2+LQ==
Connection: Upgrade
Upgrade: websocket
User-Agent: Playwright/1.29.0 (x64; arch unknown) node/19.2
Host: 127.0.0.1:39959
content-length: 0
Response:
HTTP/1.1 101 WebSocket Protocol Handshake
Upgrade: WebSocket
Connection: Upgrade
Sec-WebSocket-Accept: ZG7YduOgTTPTF8X+QAb3wvI3tiY=
content-length: 0
The WebSocket
tab shows that 81 Websocket messages where sent (I don't see an option to export them, unfortunately. But this doesn't seem to be relevant, see below).
The last displayed message was Closed by client with code 1005.
Here is the request and response when Deno executes the above script:
Request:
GET /devtools/browser/dc49d18d-00b4-4a28-9de9-92208e57363d HTTP/1.1
sec-websocket-version: 13
sec-websocket-key: HGN6NYPMLjBIFVkwSfW69g==
connection: Upgrade
upgrade: websocket
user-agent: Playwright/1.28.1 (x64; arch unknown) node/16.17
accept: */*
accept-language: *
accept-encoding: gzip, br
host: 127.0.0.1:39959
content-length: 0
Response:
HTTP/1.1 101 WebSocket Protocol Handshake
Upgrade: WebSocket
Connection: Upgrade
Sec-WebSocket-Accept: sza7yPet20xGLEbspLeZ8ZQxGDg=
content-length: 0
The WebSocket
tab shows that 0 Websocket messages where sent.
The last displayed message was Closed by client with code 1006.
There are a few differences I see:
1) The user agent is different
With Node.js the user agent is Playwright/1.29.0 (x64; arch unknown) node/19.2
.
With Deno the user agent is Playwright/1.28.1 (x64; arch unknown) node/16.17
.
The question is, why does Deno not use the latest version of Playwright? And why is the Node.js version so old with Deno?
2) The Websocket exit code:
With Node.js the exit code is 1005
.
Here is a list of Websocket status codes, and their meaning. For 1005
it says:
# 1005 CLOSED_NO_STATUS
1005 is a reserved value and MUST NOT be set as a status code in a Close control frame by an endpoint. It is designated for use in applications expecting a status code to indicate that no status code was actually present.
With Deno the exit code is 1006
, which means (also from the above link):
# 1006 CLOSE_ABNORMAL
1006 is a reserved value and MUST NOT be set as a status code in a Close control frame by an endpoint. It is designated for use in applications expecting a status code to indicate that the connection was closed abnormally, e.g., without sending or receiving a Close control frame.
So something doesn't seem to work properly.
Does anybody have any clue what the reason for 1) and 2) could be?
PS: If anybody wants to see themselves what Websocket messages are sent, here is how:
mitmweb --mode reverse:http://127.0.0.1:$WEBSOCKET_PORT
($WEBSOCKET_PORT
needs to be replaced with the port of $WEBSOCKET_ADDRESS
)$WEBSOCKET_ADDRESS
in the script to 127.0.0.1:8080
http://http://127.0.0.1:8082/
, to open the mitmproxy
web UII've tried running my test script with the --inspect-brk
option (thanks @uasi, for mentioning that option!), which lead to some odd discoveries.
First, I'll list the steps to reproduce what I did:
deno run --unstable --allow-all --inspect-brk main.ts
chrome://inspect
in Chromium to connect the debuggerTargets
click on inspect
near the Deno
targetPause on caught exceptions
Now script execution stops on any caught exceptions.
The following two exceptions seem odd to me:
Cannot find module 'bufferutil'
Cannot find module 'utf-8-validate'
Could these lead to problems?
Besides that, there are some breakpoints/exceptions where I have no clue what they are about.
And there is (of course) the exception that is printed to the terminal.
With the above approach it should be not too difficult to find the location where things break.
Today I continued to debug "Playwright via Websockets".
(Btw., I'm describing what I do in so much detail partly so I myself can reproduce it, e.g., if I have to stop and come back a few weeks/months later).
The following two exceptions seem odd to me:
- Cannot find module 'bufferutil'
- Cannot find module 'utf-8-validate'
Node.js also has a --inspect-brk
option. The above exceptions are also raised with Node.js, so don't seem to be a problem.
While debugging, it makes sense to disable the timeout of the chromium.connectOverCDP
call. The endpoint can also be simplified:
const browser = await chromium.connectOverCDP("http://127.0.0.1:8080/", { timeout: 0 });
(Defining the endpoint as the address of the HTTP server breaks support for the "man-in-the-middle" proxy. If the proxy is needed, then the Websocket address needs to be supplied directly to Playwright)
With Deno the user agent is Playwright/1.28.1 (x64; arch unknown) node/16.17. The question is, why does Deno not use the latest version of Playwright? And why is the Node.js version so old with Deno?
Importing Playwright via import { chromium } from 'npm:playwright@^1.29.1';
and adding the --reload
option to Deno updates Playwright to the last version (maybe --reload
would have been enough).
However, that doesn't seem to change anything
Here is where the exception is triggered (playwright-core/src/server/transport.ts#L82):
transport._ws.on('unexpected-response', (request: ClientRequest, response: IncomingMessage) => {
const chunks: Buffer[] = [];
const errorPrefix = `${url} ${response.statusCode} ${response.statusMessage}`;
response.on('data', chunk => chunks.push(chunk));
response.on('close', () => {
const error = chunks.length ? `${errorPrefix}\n${Buffer.concat(chunks)}` : errorPrefix;
progress?.log(`<ws unexpected response> ${error}`);
reject(new Error('WebSocket error: ' + error));
transport._ws.close();
});
});
The value of the error that this code triggers is:
"ws://127.0.0.1:8080/devtools/browser/dc49d18d-00b4-4a28-9de9-92208e57363d 101 Switching Protocols"
transport._ws
is an instance of WebSocket
defined in the ws package.
Here are the API docs for the unexpected-response
event (not really anything useful).
The request
argument is https://nodejs.org/api/http.html#class-httpclientrequest, and the response
argument is https://nodejs.org/api/http.html#class-httpincomingmessage.
Here is the location where the unexpected-response
event is emitted.
This code is within an event handler that handles responses for the WebSocket client (which are expected to be redirects).
The odd thing is, that the response has the status code 101 Switching Protocols
(see error above, or via debugger).
This is odd, because below the response handler is a upgrade
event handler (see here).
It seems, the response is somehow wrongly categories as regular response, instead of as an upgrade...
So I guess, the 'NPM compat' code does not implement https://nodejs.org/api/http.html#event-upgrade correctly.
This can be reproduced via the example of Event: 'upgrade' (replace the import at the top with import * as http from "https://deno.land/std@0.170.0/node/http.ts";
, to make it compatible with Deno).
This example is definitely a much better target to debug, than Playwright directly.
I tried to figure out why the upgrade
event isn't emitted, but I'm not familiar with the code base of Deno/Node.js, and I'm not really a TypeScript programmer. It seems, that the upgrade
event isn't implemented at all for the HTTP client (it is, however, for the HTTP server).
I think it makes sense to open a separate issue for this (to eliminate all the unnecessary information). It should be linked below this comment.
Hey @d4h0 thanks for detailed examination, this should be enough to debug the problem on our side.
I can answer some questions outright:
With Deno the user agent is Playwright/1.28.1 (x64; arch unknown) node/16.17.
The question is, why does Deno not use the latest version of Playwright? And why is the Node.js version so old with Deno?
That's probably because you used unversioned import and you had older version of Playwright cached, as you noticed --reload
flag fixed the problem (ie. Deno pulls the latest Playwrigth package available). The node/16.17
is because that's what we were targeting in our Node compat layer, but it should most likely be updated as we target latest LTS version these days - opened https://github.com/denoland/deno_std/issues/3057 to fix it.
This can be reproduced via the example of Event: 'upgrade' (replace the import at the top with import * as http from "https://deno.land/std@0.170.0/node/http.ts";, to make it compatible with Deno).
This seems like main crux of the problem and we received reports that Vite is not working properly with Deno too; it uses HTTP upgrade as well. We are currently working on a rewrite of our HTTP server that should help us fix this problem. Obviously if we could fix up the problem before rewrite of HTTP server lands that would be preferable. @kt3k could you take a look at this problem?
EDIT: Ooops, it seems the problem is with the HTTP client, not HTTP server. So the Vite problem is not related to this one.
@bartlomieju: Thank you for the feedback!
Ooops, it seems the problem is with the HTTP client, not HTTP server. So the Vite problem is not related to this one.
Is there also a rewrite of the HTTP client planned?
In theory, it shouldn't be too difficult to add HTTP upgrade support to the client. In practice, I gave up because I'm too unfamiliar with the code base and TypeScript itself (also, because I already spent three full days debugging this).
I think, Deno is in almost every way better than Node.js. Unfortunately, Playwright is essential for me, and it is too much work to maintain systems written for two different JavaScript runtimes. So, at least for me, the lack of Playwright on Deno is blocker for switching to Deno (besides Playwright, everything I need can be replaced with something else that works on Deno).
@d4h0 if that is your only blocker, try https://github.com/kt3k/deno-bin
Then package.json can be upgraded to use deno where it makes sense. And e2e test can run via node. At least until deno is fully compatible.
My biggest issue with deno is my fingers will still type node and npm. Should be easy to have deno compile bin to run package.json scripts & alias deno & translate command line params when running node & npm from folder :)
@tamusjroyce: Thanks for the tip, this looks pretty interesting.
The biggest advantage of Deno over Node.js (for me) is the fact that you can embed Deno easily into Rust applications (via the deno_core Rust library).
Currently, I'm using Node.js via a homegrown RPC utility. Deno would allow me to integrate much more ergonomically with my applications (also because Deno natively runs Wasm, so I even can run Rust directly in Deno, which makes many things easier).
My main goal for Playwright on Deno is to somehow make it possible to ergonomically use Playwright from Rust (there is already playwright-rust, but it doesn't support the latest Playwright version, and embeds a Zip binary of a Node.js application, which is unzipped at runtime, which I don't really like).
That being said, I probably will still start using kt3k/deno-bin
with my current RPC-setup to get used to and benefit from the better tooling that Deno offers. Thanks again!
With https://github.com/denoland/deno/pull/19412 is this closable?
The steps in https://github.com/denoland/deno/issues/16899#issuecomment-1364675410 seem working now! But it looks too hacky to me (involving a lot of manual copy pasting during test run). Is there any reasonable option to start playwright with websocket configuration? cc @d4h0
The steps in https://github.com/denoland/deno/issues/16899#issuecomment-1364675410 seem working now!
That sounds fantastic!
Unfortunately, I didn't have any time yet, to play with Playwright on Deno after the recent changes related to this issue.
But it looks too hacky to me (involving a lot of manual copy pasting during test run). Is there any reasonable option to start playwright with websocket configuration?
It should be possible to run Playwright similarly to how you'd run Playwright on Node.js.
For example, BrowserType::launchServer should be usable to start the browser and to get a WebSocket address via BrowserServer::wsEndpoint.
After that, we should be able to use BrowserType::connect to connect to the WebSocket endpoint (instead of using BrowserType::connectOverCDP, as the last test script does).
For example:
import { chromium } from 'npm:playwright';
async function main() {
const browserServer = await chromium.launchServer();
const wsEndpoint = browserServer.wsEndpoint();
const browser = await chromium.connect(wsEndpoint);
const page = await browser.newPage();
await page.goto('http://example.com');
const title = await page.title();
console.log(`=> Page title: ${title}`)
await browser.close();
};
main()
(Save to main.js
and run via deno run --unstable --allow-all main.js
. --unstable
might not be required anymore, I'm not sure).
Unfortunate, that fails with the following error:
Uncaught TypeError: Cannot read properties of undefined (reading 'on') Failed to launch browser
The pipe
in the error message made me think, that the issue might be the same as with the default way to run Playwright (Playwright on Node.js normally connects to the browser via pipes, not via a WebSocket), so I tried to disable this "pipe mode" via:
const browserServer = await chromium.launchServer({ignoreDefaultArgs: ["--remote-debugging-pipe"]});
...but this doesn't seem to change anything, unfortunately.
The error is triggered by the first line (that contains chromium.launchServer
) – so a workaround might be:
Both options are not great, however, so a fix for the above error would be the ideal solution.
Unfortunately, I don't have more time to play with this, right now.
The code in @d4h0 's code actually can partially work today when the option useWebSocket: true
is passed to chromium.launchServer
. This option doesn't appear documented, but it almost gets things working out of the box.
The problem is that, when the browser is navigated, such as in await page.goto('http://example.com');
, the page will almost immediately close and the following error presents itself in the console:
error: Uncaught (in promise) Error: page.goto: Browser has been closed
==== Closed by ====
at Proxy.emit (ext:deno_node/_events.mjs:382:28)
at Object.action (ext:deno_web/02_timers.js:153:11)
at handleTimerMacrotask (ext:deno_web/02_timers.js:67:10)
at eventLoopTick (ext:core/01_core.js:189:21)
This is effectively covering up the actual error which is somehow leading to the page getting closed by Playwright:
Uncaught RangeError: Invalid WebSocket frame: RSV2 and RSV3 must be clear
Unfortunately, I didn't copy the whole stack trace to my notes, and I lost track of where it originally occurred. Might have been from the "Connection" class. Not sure now.
In any case, I'm pretty sure that this seems indicative that there's something either incorrect with Deno's WebSocket implementation or with Playwright's handling of its Node shim for WebSocket. Though it's possible to successfully use some methods like evaluate
on the initial blank page, any navigation causes this issue.
In the meantime, I came up with a way to automate the connectOverCDP
approach that seems to get Playwright to work with Chromium under Deno with seemingly all basic functionality working properly.
import { chromium } from 'playwright';
// Since there's no good way for us to get the
// websocket that Chromium randomly generates,
// we must generate our own and keep track of it.
const port = getRandomPortNum();
const browserServer = await chromium.launchServer({
// This option isn't documented, but sets up the server
// to listen to a debugging port. Unfortunately, this doesn't
// seem to work out of the box, at least in Deno. By itself,
// a window can be launched, but encounters an error:
// `Uncaught RangeError: Invalid WebSocket frame: RSV2 and RSV3 must be clear`.
// So instead, we're going to connect to a devtools websocket
// later. However, I'm still including this because it
// allows us to await the browserServer object.
//
// If this option isn't available to you with your
// version of Playwright, you can remove it if you
// also remove the `await` keyword before `chromium.launchServer`.
// However, you won't have access to a BrowserServer
// object if you go down that route.
useWebSocket: true,
// Pass in our randomly generated port.
args: [`--remote-debugging-port=${port}`],
// Prevent Playwright from overriding our custom port number.
// The "--remote-debugging-pipe" flag is already removed by
// `useWebSocket`, but I left it here in case you want to
// remove that option and not await `chromium.launchServer`.
ignoreDefaultArgs: ["--remote-debugging-pipe", "--remote-debugging-port=0"],
// For demonstration purposes
headless: false,
});
// Even though we've awaited the browser server, sometimes
// the endpoint for information on the devtools websocket
// still isn't totally available. This seems to happen
// 1/10 attempts. I wrapped our call to connect in a function
// that will retry until the socket is available. There's
// probably a better way, but I wanted to make sure this
// was more reliable.
const browser = await tryWithBackoff({
// I've tried the approach in the following article and it
// simply isn't usable (yet) in Deno:
// https://playwright.dev/docs/api/class-browsertype#browser-type-launch-server
//
// Instead, we're going to let Playwright ask Chromium for the
// devtools websocket endpoint.
fn: () => chromium.connectOverCDP(`http://localhost:${port}`)
});
const page = await browser.newPage();
await page.goto('https://duck.com');
const title = await page.title();
console.log(`=> Page title: ${title}`)
await new Promise(resolve => setTimeout(resolve, 5000));
await browserServer.close();
Deno.exit();
/**
* This is finds a random port number that is not being used by something else.
**/
function getRandomPortNum (): number {
const MIN_PORT_NUM = 1024;
const MAX_PORT_NUM = 65535;
const portNum = Math.ceil(Math.random() * ((MAX_PORT_NUM - 1) - MIN_PORT_NUM + 1) + MIN_PORT_NUM + 1);
try {
const server = Deno.listen({ port: portNum });
server.close();
return portNum;
} catch (e) {
if (e.name !== 'AddrInUse') throw e;
return getRandomPortNum();
}
}
/**
* Tries to execute a function and retries (with backoff and timeout) if an error occurs.
**/
async function tryWithBackoff (args: {
fn: () => any;
delay?: number;
timeout?: number;
startedAt?: number;
error?: Error;
}): Promise<any> {
const { fn, delay, timeout, startedAt, error } = {
delay: 0,
timeout: 30000,
startedAt: Date.now(),
...args,
};
await new Promise(resolve => setTimeout(resolve, delay));
if ((Date.now() - startedAt) > timeout) {
throw (error || new Error('Function call timed out'));
}
try {
return await fn();
} catch (error) {
console.error(error);
return tryWithBackoff({
fn,
delay: delay + 1000,
timeout,
startedAt,
error,
});
}
}
Just trying playwright out myself, in Deno 1.38.0, when running:
deno run -A npm:playwright@1.39.0 install
I get the follow error: process.geteuid is not a function
, full error:
error: Uncaught TypeError: process.geteuid is not a function
at file:///Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.39.0/lib/transform/compilationCache.js:46:91
at Object.<anonymous> (file:///Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.39.0/lib/transform/compilationCache.js:47:3)
at Object.<anonymous> (file:///Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.39.0/lib/transform/compilationCache.js:223:4)
at Module._compile (node:module:733:34)
at Object.Module._extensions..js (node:module:747:10)
at Module.load (node:module:658:32)
at Function.Module._load (node:module:539:12)
at Module.require (node:module:677:19)
at require (node:module:791:16)
at Object.<anonymous> (file:///Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.39.0/lib/transform/transform.js:20:25)
Running on MacOS btw, so this won't affect Windows, but will for Linux users too, looks like this is the cause...
Deno 1.38.2 now implements process.geteuid()
so no longer get the above, but the following...
deno run -A npm:playwright@1.40.0 install
Downloading Chromium 120.0.6099.28 (playwright build v1091) from https://playwright.azureedge.net/builds/chromium/1091/chromium-mac.zip
Failed to install browsers
Error: Failed to download Chromium 120.0.6099.28 (playwright build v1091), caused by
TypeError: cp.send is not a function
at downloadBrowserWithProgressBarOutOfProcess (file:///Users/mgibson/ahx/ahx_lib/node_modules/.deno/playwright-core@1.40.0/node_modules/playwright-core/lib/server/registry/browserFetcher.js:118:6)
at downloadBrowserWithProgressBar (file:///Users/mgibson/ahx/ahx_lib/node_modules/.deno/playwright-core@1.40.0/node_modules/playwright-core/lib/server/registry/browserFetcher.js:51:17)
at eventLoopTick (ext:core/01_core.js:178:11)
at async Registry._downloadExecutable (file:///Users/mgibson/ahx/ahx_lib/node_modules/.deno/playwright-core@1.40.0/node_modules/playwright-core/lib/server/registry/index.js:738:5)
at async Registry.install (file:///Users/mgibson/ahx/ahx_lib/node_modules/.deno/playwright-core@1.40.0/node_modules/playwright-core/lib/server/registry/index.js:687:9)
at async t.<anonymous> (file:///Users/mgibson/ahx/ahx_lib/node_modules/.deno/playwright-core@1.40.0/node_modules/playwright-core/lib/cli/program.js:113:7)
seems ChildProcess is missing send()
. (https://github.com/denoland/deno/issues/12879)
But also, after installing browsers via node and manually caching a package, then attempting to run tests...
yarn playwright install
deno cache npm:@playwright/test
deno run -A deno run -A npm:playwright@1.40.0 test
it did attempt to run the entire test suite, but with lots of warnings...
Warning: Not implemented: process.on("message")
Warning: Not implemented: process.on("disconnect")
Error: worker process exited unexpectedly (code=0, signal=null)
and then successfully started the test report server and opened the report in my browser.
So, I wonder if it's actually quite close to working once message
and disconnect
events are supported. Do we know if there a fundamental issue preventing those, or simply a case of not got round to it yet?
(EDIT: is this related to https://github.com/denoland/deno/issues/16298 ?)
So, I wonder if it's actually quite close to working once message and disconnect events are supported. Do we know if there a fundamental issue preventing those, or simply a case of not got round to it yet?
No fundamental issues, we are actually eye-balling implementing these APIs in the nearest future and they are required for several other packages too (eg. Next.js).
It seems like having the pipe
launchOption in Puppeteer set to true has the same result.
See https://github.com/lucacasonato/deno-puppeteer/issues/56
It seems we are getting closer, with Deno 1.40.0 ...
deno run -A npm:playwright@1.41.1 install
now downloads the browsers, but the process seems to just hang after fetching them all!
Minor detail, Deno dumps out a deprecation warning about it's own node compat layer!:
warning: Use of deprecated "Deno.FsFile.rid" API. This API will be removed in Deno 2.
Stack trace:
at ext:deno_node/_fs/_fs_open.ts:78:96
hint: Use `Deno.FsFile` methods directly instead.
hint: It appears this API is used by a remote dependency. Try upgrading to the latest version of that dependency.
For your information, puppeteer now works fine if you substitute the "ws" node package with the native WebSocket from deno (https://github.com/denoland/deno/issues/20179). Maybe playwright is similar I didn't try yet. Unfortunately I have no idea what's the procedure to fix this properly without resorting to vendoring+patching.
@guillaume86 How would you fix it with vending + patching? If I understand correctly, the fix is to have playwright use Deno's global WebSocket instead of ws
? I tried and failed to figure out where to perform the substitution. Have you been able to get something working?
@guillaume86 How would you fix it with vending + patching? If I understand correctly, the fix is to have playwright use Deno's global WebSocket instead of
ws
? I tried and failed to figure out where to perform the substitution. Have you been able to get something working?
I meant making a local copy of the playwright lib and change the source code at your will (basically a fork). It's not great as you imagine. Maybe monkey-patching could work too, I gave up using deno with playwright for the moment...
@d4h0 @bartlomieju does this library can solve this? https://github.com/gildas-lormeau/simple-cdp/
https://github.com/denoland/deno/issues/16899#issuecomment-1910625997
deno run -A npm:playwright@1.41.1 install
now downloads the browsers, but the process seems to just hang after fetching them all!
It is due to a bug in node:child_process.fork()
, which will be fixed soon (https://github.com/denoland/deno/issues/24756)
The current blocker is the lack of support for extra stdio pipes (so more than just stdin, stdout, stderr). The main complication is in how these pipes are consumed by the child process. If the child process is a deno subprocess, then generally you would create a new socket with a specific FD and use that as the child end of the pipe. For instance,
// child.js
import net from "node:net";
const sock = new net.Socket({ fd: 4 }); // assuming the parent process opened a pipe on fd 4
The issue with that is that we don't currently support interacting with resources using raw file descriptors, as it's a larger architectural change and requires careful thought about how it interacts with the permissions system. I initially thought that this may be a blocker for this issue.
Digging into playwright specifically, however, I think we can sidestep this issue as the child processes are actually native code. So for chromium, for instance, playwright spawns a child process with pipes at fds 3 and 4. Then, chromium opens pipes on those file descriptors (from C++). Since the child process isn't JS, we don't have to solve the more general issue of using raw FDs in deno, so I believe I can make progress on this issue.
This should now be working on canary, and will be in the next release (only on macOS and linux, for now).
import { chromium } from "npm:playwright";
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
async function main() {
const browser = await chromium.launch({
headless: false,
});
const page = await browser.newPage();
await page.goto("http://deno.com");
await sleep(2000);
await browser.close();
}
if (import.meta.main) {
await main();
}
https://github.com/user-attachments/assets/189c5ec2-842e-4e98-bac2-74bbce5af8e5
@nathanwhit You are a hero! 🦸🏻
💪🏻 💪🏻 💪🏻
Just trying out Playwright 1.46.1 on Deno 1.46.1, which is a strange coincidence in itself!
Anyway, I can confirm that playwright install
now works perfectly.
But I can't get my test suite to run...
Task test deno run --allow-all npm:playwright test
Error: Cannot find module '@playwright/test'
Require stack:
- /Users/mgibson/ahx/ahx_lib/playwright.config.js
- /Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.46.1/lib/transform/transform.js
- /Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.46.1/lib/common/config.js
- /Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.46.1/lib/reporters/json.js
- /Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.46.1/lib/runner/reporters.js
- /Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.46.1/lib/runner/runner.js
- /Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.46.1/lib/program.js
- /Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.46.1/cli.js
at Function.Module._resolveFilename (node:module:608:15)
at Function.resolveFilename (/Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.46.1/lib/transform/transform.js:212:36)
at Function.Module._load (node:module:486:27)
at Module.require (node:module:674:19)
at require (node:module:798:16)
at Object.<anonymous> (/Users/mgibson/ahx/ahx_lib/playwright.config.js:1:1)
at Object.<anonymous> (/Users/mgibson/ahx/ahx_lib/playwright.config.js:86:4)
at Module._compile (node:module:736:34)
at Module.f._compile (/Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.46.1/lib/utilsBundleImpl.js:16:994)
at Module._extensions..js (node:module:757:11)
at Object.i.<computed>.ut._extensions.<computed> (/Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.46.1/lib/utilsBundleImpl.js:16:1010)
at Module.load (node:module:655:32)
at Function.Module._load (node:module:523:13)
at Module.require (node:module:674:19)
at require (node:module:798:16)
at requireOrImport (/Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.46.1/lib/transform/transform.js:192:20)
at loadUserConfig (/Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.46.1/lib/common/configLoader.js:96:83)
at loadConfig (/Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.46.1/lib/common/configLoader.js:101:28)
at loadConfigFromFileRestartIfNeeded (/Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.46.1/lib/common/configLoader.js:258:16)
at runTests (/Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.46.1/lib/program.js:194:76)
at t.<anonymous> (/Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.46.1/lib/program.js:54:7) {
code: "MODULE_NOT_FOUND",
requireStack: [
"/Users/mgibson/ahx/ahx_lib/playwright.config.js",
"/Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.46.1/lib/transform/transform.js",
"/Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.46.1/lib/common/config.js",
"/Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.46.1/lib/reporters/json.js",
"/Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.46.1/lib/runner/reporters.js",
"/Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.46.1/lib/runner/runner.js",
"/Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.46.1/lib/program.js",
"/Users/mgibson/Library/Caches/deno/npm/registry.npmjs.org/playwright/1.46.1/cli.js"
]
}
Here is my playwright.config.js... https://github.com/jollytoad/ahx/blob/playwright-deno/playwright.config.js
@playwright/test
is mapped in my deno.json... https://github.com/jollytoad/ahx/blob/playwright-deno/deno.json#L31
I've deliberated removed node and related tools, so it should be running pure deno.
Any ideas? Anyone else got a playwright test suite to run yet on the recent releases?
@jollytoad Hello! I dug into this and it seems we have a bug in this case, specifically when you aren't using a node modules directory (https://github.com/denoland/deno/issues/25189).
To work around this for now, you can add
"nodeModulesDir": true
to your deno.json
to opt into a node_modules
folder.
You'll also need to make sure that the dependencies get cached for playwright.config.js
, before you run the tests.
You can do this with just deno cache playwright.config.js
(I would put it in your task probably, just before running playwright test
)
Testing this, it does seem while the tests do run correctly, the process doesn't always exit cleanly, and I sometimes have to exit with ctrl-c
– I haven't looked into what's happening there yet.
@nathanwhit Thank you so much for pushing this forward! Is there a write up anywhere of what it will take to get playwright working on Windows? Unfortunately, many of the testers on our team use it, and so I can't fully migrate us until it lands.
@nathanwhit thanks for the advice, I can confirm my tests run after setting nodeModulesDir
and also running deno cache
to ensure the node_modules is populated as expected. And I also get the problem that the test suite appears to hang at the end, I have to manually hit ctrl+c
, after which it launches the results web app. I've found that hitting ctrl+c
again to then kill the server seems to return me to the shell prompt, but the web server appears to continue to run in the background!
Hi,
I'm trying to use Playwright via the new NPM compatibility layer, but it fails with:
Uncaught TypeError: browserType.launch: Cannot read properties of undefined (reading 'on')
The following snippet can be used to reproduce the error:
After running the above via
deno run --unstable --allow-all main.ts
, the following message is displayed:Executing
deno run --unstable --allow-all npm:playwright install
downloads the required binaries.Executing
deno run --unstable --allow-all main.ts
again, leads to the above-mentioned error:Playwright
/Deno
was already discussed in https://github.com/denoland/deno/issues/16298, but I thought it makes sense to open an issue that focuses only onPlaywright
. There was also some discussion in thePlaywright
repo: https://github.com/microsoft/playwright/issues/3146