Playwright: run e2e tests locally in a CI container

[fungairino]

Currently we don't run e2e tests the same way locally as we do in the github workflow. Ideally we could spin up a container and run it in exactly the same way as it will run in CI (in a linux latest ubuntu image)

Some options:

Docker script https://github.com/lblod/ember-rdfa-editor/blob/f31ba5c4714a1dc59609be3402c8c6fd63914abf/scripts/e2e.sh

Or

Use act https://nektosact.com/

https://github.com/microsoft/playwright-python/issues/796 https://github.com/qemus/qemu-docker

[fungairino]

Using act (https://nektosact.com) seemed promising, but it is way too slow for iterative testing, and I wasn't able to get a full e2e run working. It usually got stuck at the build extension step.

The command I used: act -W '.github/workflows/ci.yml' -j 'end-to-end-tests' --container-architecture linux/amd64

I also had to modify the ci.yml to conditionally install xvfb:

      - name: Install xvfb
        run: apt-get update && apt-get install xvfb -y
        if: ${{ env.ACT }}

[fungairino]

I have a "working" docker build for end to end tests, the only problem I'm seeing now is that the tests are really slow, leading to a lot of time outs.

Here is the docker file I used:

playwright.dockerfile

FROM --platform=linux/amd64 mcr.microsoft.com/playwright:v1.44.0-jammy AS tester
WORKDIR /app
COPY . .

#TODO: load these from the local .env.development file
#ENV CI=1 # uncomment to avoid opening the report if the tests fail
ENV CHROME_MANIFEST_KEY=<fill in>
ENV CHROME_EXTENSION_ID=mpjjildhmpddojocokjkgmlkkkfjnepo
ENV MV=3
ENV SHADOW_DOM=open
ENV REQUIRE_OPTIONAL_PERMISSIONS_IN_MANIFEST=1

ENV E2E_TEST_USER_EMAIL_UNAFFILIATED=extension-e2e-test.unaffiliated@pixiebrix.test
ENV E2E_TEST_USER_PASSWORD_UNAFFILIATED=<fill in>

ENV DATADOG_APPLICATION_ID=<fill in>
ENV DATADOG_CLIENT_TOKEN=<fill in>

RUN npm ci
RUN npm run build:webpack
 # I thought that the playwright docker image would come with the browsers installed, but doesn't seem to be the case.
RUN npx playwright install chrome msedge
CMD [ "npm", "run", "test:e2e"]

Then build with: DOCKER_DEFAULT_PLATFORM=linux/amd64 docker build -f playwright.dockerfile --progress=plain --tag=e2e . Then run: DOCKER_DEFAULT_PLATFORM=linux/amd64 docker run -it --rm --ipc=host e2e

(not sure if "DOCKER_DEFAULT_PLATFORM", is needed)

[fungairino]

It's possible the latency is due to the emulation needed to run the container in amd64 on an M1 chip ARM Mac laptop.

Another option is to download and install Ubuntu on parallels, and run it there.

[github-actions[bot]]

This issue will be closed in 7 days unless the stale label is removed, or a comment is added to the issue.