[Bug]: Playwright tests are passing locally but failing in GitHub actions (retried for 3 times the same result)

[pavaneeshwar7077]

Version

1.36.1

Steps to reproduce

This is the workflow file we use for installation :

name: CI/CD for Test To Dev

permissions:
  id-token: write
  contents: read

on:
  pull_request:
    branches:
      - dev

jobs:
  build:
    runs-on: ubuntu-latest
    environment: test

    steps:
    - name: Checkout code
      uses: actions/checkout@v4

    - name: Setup Node
      uses: actions/setup-node@v4
      with:
        node-version: '18'

    - name: Create .env File
      run: |
        cd altius-nx
        echo "ENV_REF_NAME=$ENV_REF_NAME" >> .env
        echo "EMAIL=$EMAIL" >> .env
        echo "PASSWORD=$PASSWORD" >> .env
        echo "BASE_URL=$BASE_URL" >> .env
      env:
        ENV_REF_NAME: ${{ secrets.ENV_REF_NAME }}
        EMAIL: ${{ secrets.EMAIL }}
        PASSWORD: ${{ secrets.PASSWORD }}
        BASE_URL: ${{ secrets.BASE_URL }}

    - name: Install yarn
      run: |
        cd altius-nx
        npm install -g yarn

    - name: Install Project Dependencies
      run: |
        cd altius-nx
        yarn install

    - name: Install Playwright Browsers
      run:  |
        cd altius-nx
        yarn playwright install --with-deps

    - name: Run Playwright tests
      run: |
        cd altius-nx
        yarn test --grep @field

    - uses: actions/upload-artifact@v4
      if: always()
      with:
        name: playwright-report
        path: altius-nx/playwright-report/index.html
        retention-days: 7

Expected behavior

All the test cases should pass when executed in github actions, they are passing in local

Actual behavior

Failing in pipeline ( 1 test case)

Additional context

No response

Environment

Node : 18

[dgozman]

We need more information to act on this issue. Could you please provide a full repro that we can run locally? Additionally, I'd recommend to update Playwright to the latest version.

[Sudipt2001AltiusHub]

@dgozman we are using these parameters for the config file -

import { defineConfig } from '@playwright/test';

export default defineConfig({ testDir: 'apps/altius-nx/src/tests', timeout: 600000, reporter:'html', retries:2, forbidOnly: true, workers:1, use: { headless: true, screenshot: 'only-on-failure', viewport: { width: 1280, height: 720 }, actionTimeout:120000, }, });

The tests are passing locally -

For GitHub actions, 1 test case is failing. If we run the test case separately on local machine, it is passing. If we skip the test case using tests.skip, all other tests are passing, and the build is successful. We are not able to identify the issue since the test is passing on local machine.

We are debugging the flaky nature of the test case, but we expect the same result in GitHub actions as well.

[dgozman]

@Sudipt2001AltiusHub Unfortunately, we cannot repro by just looking at your config file 😄 I'd recommend using our debugging tools, for example record a trace on CI and explore it in the Trace Viewer, to see what went wrong.