[Feature] Add back support for docker and VNC

[pastelsky]

This is in furtherance of the communication on this thread with @aslushnikov — https://github.com/microsoft/playwright/pull/20691

Recently, support for spinning up official docker images using playwright CLI was dropped from Playwright. While it was never really documented, it was quite helpful to us in various ways.

Firstly, our use case — we've been leaning onto playwright (component testing) as a visual regression solution that we hope to use across several large applications at our company.

Playwright appealed to use primarily due to its clean design, and multi-browser support, but also because it seemed to have sound opinions on the "operational" characteristics of a test suite — with first-class support for parallel runs & a server abstraction that could be used to separate application compilation / serving from the rendering of the application. Moreover, it's philosophy seemed to encourage local debuggability of tests. For our usage (and hopefully others too) —

1. Consistent environment is key: For any serious VR test suite, a consistent rendering environment & reproducability is a necessity. Having browser runtimes in docker that tests can connect to is thus essential for a local dev loop & CI.

2. First class support for debugging through VNC: It was exciting to see support for VNC added — again, this has the possibility of upping the local debugging experience significantly. This meant that one could now run visual regression tests in debug/headed mode, and be able to step through / inspect visually — all while still running browsers in a consistent docker environment.

3. Scalability: I could be wrong, but it seemed to me that the playwright docker image was headed towards a solution that would essentially allow deploying playwright browsers as a service. With a single URL proxy to connect to browser WS, it would be possible to deploy these on serverless solutions like AWS container images, essentially farming out tests to 100s parallel browsers at significantly lower costs and better reliability than third-party solutions.

In my opinion, this batteries-included approach can be a key differentiator for playwright — as most other automation frameworks essentially force you today into making a compromise between debuggability/reproducability/scalability.

[pirate]

Has anyone tried using https://github.com/linuxserver/docker-baseimage-kasmvnc (or something similar) + playwright to run a browser-in-docker that attaches to a virtual X11 display viewable over VNC?

I haven't had a chance to try it myself yet but I suspect someone could get it working. Kasmvnc's docker image also includes a webserver with an in-browser VNC client to view the virtual X11 display.

Further reading:

• https://notes.dt.in.th/CodespacesDisplayServer
• https://aerokube.com/moon/latest/#moon-pod-contents (moon includes a VNC-server pod that allows viewing playwright browser sessions)

[Janpot]

I'm primarily interested in reducing the overhead involved in updating snapshots in visual regression testing. The official solution (reinstall and rebuild your project in a docker container) is so cumbersome when you have snapshots for multiple architectures.

[markbrockhoff]

I totally agree with @Janpot. It would be great to have the option to just run the tests inside a platform independent container. Having to create a container, installing all project dependencies and then running all tests in there for visual regression tests is just painful. Especially as this has to be done locally and in CI. I really liked the idea of having a container sub-command within the cli to simplify this like in #20691.

I don't neccessarily need a VNC connection to see the test while they're running. It could be nice to expose the trace viewer ui from the container to directly work within it but that could be added at a later stage. For a first iteration it would be awesome to have a command that spins up a docker container, connects to the browser within it and runs the tests inside it. This would give us the option to write tests locally (possibly ignoring screenshot tests), then running them inside the container (headless of course) for creating the visual regression images and then running the tests inside the container again during CI.

@aslushnikov please keep on working on this feature.

[pavelfeldman]

@aslushnikov please keep on working on this feature.

It is pretty much stalled due to the lack of demand at this point. This issue only has a dozen of upvotes, which is too little to move it up in our list.

[erfanimani]

+1

I'm working on a React application that behaves differently inside Playwright than inside the browser that runs directly on my host. It's pretty much impossible to debug this unless I run Playwright on my host instead (which defeats the whole point of running containers in the first place — I generally refrain from running any project dependency on my host directly). It would be good to be able to connect through VNC to Playwright UI.

edit:

For others coming across this, it was actually pretty easy to add VNC support into the Playwright image:

FROM mcr.microsoft.com/playwright:v1.36.2-jammy

ARG DEBIAN_FRONTEND=noninteractive

RUN apt-get update; \
    apt-get install -y x11vnc xvfb; \
    apt-get -y clean;
RUN mkdir ~/.vnc
RUN x11vnc -storepasswd {yourpassword} ~/.vnc/passwd

EXPOSE 5900

[deviantintegral]

I'm not sure doing this in Playwright is the best layer of the stack to run things in. For example, if you run playwright on the host, but also have native dependencies, then there will be conflicts I think? As well, without docker configuration Playwright may not even be on the same docker network as the app being tested.

For those interested in kasmvnc, we have them working in two different implementations with ddev. ddev is a wrapper around docker-compose for web development so much of the code could be reused in other non-ddev context:

https://github.com/deviantintegral/ddev-playwright https://github.com/julienloizelet/ddev-playwright

[crasite]

+1

I'm working on a React application that behaves differently inside Playwright than inside the browser that runs directly on my host. It's pretty much impossible to debug this unless I run Playwright on my host instead (which defeats the whole point of running containers in the first place — I generally refrain from running any project dependency on my host directly). It would be good to be able to connect through VNC to Playwright UI.

edit:

For others coming across this, it was actually pretty easy to add VNC support into the Playwright image:

FROM mcr.microsoft.com/playwright:v1.36.2-jammy

ARG DEBIAN_FRONTEND=noninteractive

RUN apt-get update; \
    apt-get install -y x11vnc xvfb; \
    apt-get -y clean;
RUN mkdir ~/.vnc
RUN x11vnc -storepasswd {yourpassword} ~/.vnc/passwd

EXPOSE 5900

Is there a working example for this? I extend this script to run playwright but it won't show the browser screen, but instead show a terminal. Here's a reference to my dockerfile

FROM --platform=linux/amd64 mcr.microsoft.com/playwright:v1.44.1-jammy 

ARG DEBIAN_FRONTEND=noninteractive

RUN apt-get update
RUN apt-get install -y x11vnc xvfb
RUN apt-get -y clean;
RUN mkdir ~/.vnc
RUN x11vnc --version

WORKDIR /app
RUN npm i -g pnpm
COPY package.json ./
RUN pnpm install -P
COPY . ./

RUN x11vnc -storepasswd secretpassword ~/.vnc/passwd 
CMD xvfb-run pnpm start & x11vnc -forever -usepw -create

EXPOSE 5900

edit:

I got it working with the following Dockerfile. I don't really know why, but I assume --auth-file has something to do with it

FROM --platform=linux/amd64 mcr.microsoft.com/playwright:v1.44.1-jammy 

ARG DEBIAN_FRONTEND=noninteractive

RUN apt-get update
RUN apt-get install -y x11vnc xvfb
RUN apt-get -y clean;
RUN mkdir ~/.vnc
RUN x11vnc --version

WORKDIR /app
RUN npm i -g pnpm
COPY package.json ./
RUN pnpm install -P
COPY . ./

RUN x11vnc -storepasswd secretpassword ~/.vnc/passwd 
CMD xvfb-run --server-num 1 --auth-file /tmp/xvfb1.auth pnpm start & \
  x11vnc -usepw -display WAIT:1 -forever -auth /tmp/xvfb1.auth

EXPOSE 5900

[edumserrano]

For those coming across this issue trying to get Playwright to run in Docker I've documented my experience in the Playwright Adventures repo, see the Docker demo:

Run Playwright tests in Docker. Also shows how to use Playwright tests UI mode in Docker. Especially helpful to eliminate screenshot differences when running the tests across different Operating Systems. For instance, if you develop in Windows but your CI runs on Linux.