[Bug]: Playwright Integration Issue: Tests Not Running in GitHub Actions with pnpm

[NidhiSharma02]

Version

1.46.1

Steps to reproduce

I want to run test on github actions but they are failing. All tests are getting passed in local in ui mode as well as in Headless Mode.

Here is playwright.yml file

   name: Playwright Tests

   on:
      push:
      branches: [ development ]
     pull_request:
      branches: [ development ]

   jobs:
     test:
    timeout-minutes: 60
    runs-on: ubuntu-20.04

    steps:
    - uses: actions/checkout@v4

    - uses: actions/setup-node@v4
      with:
        node-version: lts/*

    - name: Install dependencies
      run: npm install -g pnpm && pnpm install

    - name: Install required dependencies
      run: |
        sudo apt-get update
        sudo apt-get install -y \
          libnss3 \
          libxss1 \
          libasound2 \
          libatk1.0-0 \
          libatk-bridge2.0-0 \
          libcups2 \
          libdbus-1-3 \
          libdrm2 \
          libgbm1 \
          libgtk-3-0 \
          libnspr4 \
          libxcomposite1 \
          libxdamage1 \
          libxrandr2 \
          xdg-utils \
          libgbm-dev \
          libxshmfence-dev

    - name: Install Playwright Browsers
      run: pnpm exec playwright install --with-deps

    - name: Start application server
      run: |
        pnpm start &  # Start server in the background
        # Add a simple retry loop to check if the server is ready
        for i in {1..30}; do
          if curl -s http://localhost:3000 > /dev/null; then
            echo "Server is up!"
            break
          fi
          echo "Waiting for server..."
          sleep 10
        done

    - name: Verify server is running
      run: curl -I http://localhost:3000

    - name: Run Playwright tests with debug
      run: DEBUG=pw:browser pnpm exec playwright test
      env:
        CI: true  # Ensure Playwright knows it's running in CI
      timeout-minutes: 30  # Ensure enough time for tests

    - uses: actions/upload-artifact@v4
      if: always()
      with:
        name: playwright-report
        path: playwright-report/
        retention-days: 30

Here is config file

import path from "node:path";
import { fileURLToPath } from "node:url";
import { defineConfig } from "@playwright/test";
import dotenv from "dotenv";

const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);

// Load environment variables from .env file
dotenv.config({ path: path.resolve(__dirname, "", ".env") });

// biome-ignore lint/style/noDefaultExport: <explanation>
export default defineConfig({
    testDir: "./src/e2e",
    // testMatch: /.*.spec.ts/,
    // globalSetup: "./src/e2e/Global.setup.ts",
    // globalSetup: path.resolve(__dirname, "./src/e2e/GlobalSetup.ts"),

    fullyParallel: true,
    forbidOnly: false,
    retries: 0,
    workers: 1,
    reporter: "html",
    use: {
        baseURL: "http://localhost:3000",
        trace: "on-first-retry",
        browserName: "chromium",
        headless: true, // Set headless to true as a boolean
        launchOptions: {
            args: [
                "--headless=new", // Use the new headless mode by passing this argument to Chromium
            ],
        },
        // storageState: path.resolve(__dirname, "src/e2e/auth.json"),
    },
    // projects: [
    //  {
    //      name: "chromium",
    //      use: { ...devices["Desktop Chrome"] },
    //  },
    // ],

    projects: [
        // {
        //  name: "chromium",
        //  use: { ...devices["Desktop Chrome"] },
        // },
        {
            name: "setup",
            testMatch: "**/*.setup.ts",
        },
        {
            name: "e2e",
            dependencies: ["setup"],
            use: {
                storageState: path.resolve(__dirname, "src/e2e/auth.json"),
            },
        },
    ],
    webServer: {
        command: "pnpm start",
        url: "http://localhost:3000",
        reuseExistingServer: true,
        timeout: 120 * 1000,
        env: {
            USE_BABEL_PLUGIN_ISTANBUL: "1",
        },
    },
});

Expected behavior

I'm expecting that all test should get pass.

Actual behavior

Additional context

No response

Environment

System:
    OS: Linux 5.15 Ubuntu 22.04.3 LTS 22.04.3 LTS (Jammy Jellyfish)
    CPU: (8) x64 11th Gen Intel(R) Core(TM) i3-1125G4 @ 2.00GHz
    Memory: 719.59 MB / 3.77 GB
    Container: Yes
  Binaries:
    Node: 22.5.1 - ~/.nvm/versions/node/v22.5.1/bin/node
    Yarn: 1.22.22 - ~/.nvm/versions/node/v22.5.1/bin/yarn
    npm: 10.8.2 - ~/.nvm/versions/node/v22.5.1/bin/npm
    pnpm: 9.6.0 - ~/.nvm/versions/node/v22.5.1/bin/pnpm
  IDEs:
    VSCode: 1.93.1 - /home/nidhisharma/.vscode-server/bin/38c31bc77e0dd6ae88a4e9cc93428cc27a56ba40/bin/remote-cli/code
  Languages:
    Bash: 5.1.16 - /usr/bin/bash
  npmPackages:
    @playwright/test: ^1.46.1 => 1.47.0

[mxschmitt]

Do you have a globalSetup? Like a traditional one, not project dependencies. My suspicion is because the browser you are running has no --headless argument, that looks might be caused that you are launching a browser (launch()) somewhere manually. Would it be possible to share the entire repository with us?

Unrelated, you can remove the Install required dependencies since you are doing playwright install --with-deps - also we recommend ubuntu-24.04.

[NidhiSharma02]

Do you have a globalSetup? Like a traditional one, not project dependencies. My suspicion is because the browser you are running has no --headless argument, that looks might be caused that you are launching a browser (launch()) somewhere manually. Would it be possible to share the entire repository with us?

Unrelated, you can remove the Install required dependencies since you are doing playwright install --with-deps - also we recommend ubuntu-24.04.

yes i do have a global setup file

const __dirname = path.dirname(__filename);

setup("do login", async ({ page }) => {
    await page.goto("/", { waitUntil: "networkidle" });
    // Check if the login heading is visible
    await expect(page.getByRole("heading", { name: "Login" })).toBeVisible();

    // Fill in the login form
    await page.fill("input#email", "mayank.kumar@zeitview.com");
    await page.fill("input#password", "@Testing5624");

    // Click on the submit button
    await page.click("#login");
    await page.waitForURL("http://localhost:3000/");

    const storageState = path.join(__dirname, "./auth.json");

    // Save storage state for authentication persistence
    await page.context().storageState({ path: storageState });
});

It is not possible to share the entire github with you because it is private

[mxschmitt]

Does it work without --headless=new?

[NidhiSharma02]

No it doesn't work.

[mxschmitt]

We need more information to act on this bug report. I recommend trying if it reproduces with a new project and reducing the size of your project to a minimal one by removing files to see what is causing it. Also there might be a headless: false somewhere in your code, which would explain why the browser tries to launch headed (assumption). It looks like the CLI arguments from your screenshot in https://github.com/microsoft/playwright/issues/32712#issue-2536357162 were cut off - would it be possible to share the logs as text? Maybe you can click on "Download log archive".

I don't see a particular error on your screenshot besides browser warnings, so maybe there are more relevant logs below?

Thanks!