microsoft / playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
https://playwright.dev
Apache License 2.0
66.43k stars 3.63k forks source link

[Bug]: The dependencies installed on ubuntu-latest are not being transferred to runs-on:self-hosted #30745

Closed YusuffOzt closed 5 months ago

YusuffOzt commented 5 months ago

Version

1.44.0

Steps to reproduce

name: Run Playwright Tests

on:
  schedule:
    - cron: "0 5 * * 1-5"
  workflow_dispatch:
    inputs:
      param:
        description: "Type of test (api, ui or both)"
        required: true
        default: "both"
        type: choice
        options:
          - api
          - ui
          - both
      environment:
        description: "Environment to run the tests "
        required: true
        type: string
        default: "dev"

jobs:
  install:
    timeout-minutes: 30
    name: ⚙️ Install
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4

      - name: Cache node_modules and Playwright binaries
        uses: actions/cache@v4
        id: cache
        with:
          path: |
            test/node_modules
            ~/.cache/ms-playwright
          key: modules-${{ hashFiles('test/package-lock.json') }}

      - name: Install dependencies && Install PlayWright
        if: steps.cache.outputs.cache-hit != 'true'
        run: |
          npm ci
          sudo npx playwright install --with-deps
        working-directory: ./test

  test-api:
    name: 👀 API Tests
    needs: install
    timeout-minutes: 30
    runs-on: self-hosted
    if: ${{ github.event.inputs.param == 'api' || github.event.inputs.param == 'both' || github.event_name == 'schedule'}}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4

      - name: Download cached node_modules and Playwright binaries
        uses: actions/cache@v4
        with:
          path: |
            test/node_modules
            ~/.cache/ms-playwright
          key: modules-${{ hashFiles('test/package-lock.json') }}

      - name: Run Playwright API tests
        run: npx playwright test
        working-directory: ./test

      - name: Upload blob reports as artifact
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: blob-report-api
          path: ./test/blob-report
          retention-days: 5

      - name: Download blob reports from GitHub Actions Artifacts
        uses: actions/download-artifact@v4
        if: always()
        with:
          path: all-blob-reports
          pattern: blob-report-*
          merge-multiple: true

      - name: Merge into HTML Report
        if: always()
        run: npx playwright merge-reports --reporter html ./all-blob-reports

      - name: Upload HTML report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: api-report
          path: playwright-report
          retention-days: 5

  test-ui:
    name: 👀 UI Tests
    needs: install
    timeout-minutes: 30
    runs-on: self-hosted
    if: ${{ github.event.inputs.param == 'ui' || github.event.inputs.param == 'both' || github.event_name == 'schedule' }}
    strategy:
      fail-fast: false
      matrix:
        shardIndex: [1, 2, 3, 4, 5]
        shardTotal: [5]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4

      - name: Download cached node_modules and Playwright binaries
        uses: actions/cache@v4
        with:
          path: |
            test/node_modules
            ~/.cache/ms-playwright
          key: modules-${{ hashFiles('test/package-lock.json') }}

      - name: Run Playwright UI tests
        run: npx playwright test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
        working-directory: ./test

      - name: Upload blob reports as artifact
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: blob-report-ui-${{ matrix.shardIndex }}
          path: ./test/blob-report
          retention-days: 5

  merge-ui:
    timeout-minutes: 30
    name: 🚩 Merge UI Reports
    if: (github.event.inputs.param == 'ui' && always()) || (github.event.inputs.param == 'both' && always()) || (github.event_name == 'schedule' && always())
    needs: [install, test-ui]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4

      - name: Download blob reports from GitHub Actions Artifacts
        uses: actions/download-artifact@v4
        if: always()
        with:
          path: all-blob-reports
          pattern: blob-report-ui-*
          merge-multiple: true

      - name: Merge UI into HTML Report
        run: |
          npx playwright merge-reports --reporter html ./all-blob-reports

      - name: Upload UI HTML report
        uses: actions/upload-artifact@v4
        with:
          name: ui-report
          path: playwright-report
          retention-days: 5

I upgraded to pw to 1.44.0 version in my local. When the above GitHub YAML is executed, it does not transfer the Playwright dependencies installed on runs-on:ubuntu-latest to runs-on:self-hosted

Expected behavior

I expect the npm packages and Playwright installed on runs-on:ubuntu-latest to be transferred to runs-on:self-hosted

Actual behavior

Giving below warning; Error: browserType.launch: Executable doesn't exist at /home/vin-cicd/.cache/ms-playwright/chromium-1117/chrome-linux/chrome ╔═════════════════════════════════════════════════════════════════════════╗ ║ Looks like Playwright Test or Playwright was just installed or updated. ║ ║ Please run the following command to download new browsers: ║ ║ ║ ║ npx playwright install ║ ║ ║ ║ <3 Playwright Team ║ ╚═════════════════════════════════════════════════════════════════════════╝

Additional context

No response

Environment

playwright version:1.44.0
env: Github Action
yury-s commented 5 months ago

This is working as intended, the dependencies are not expected to be cached (also you only cache the browsers directory, not the system libraries) as they depend on the exact system you are running on. You should run npx playwright install --with-deps for each of the jobs. One way to speedup this step is to use docker image with preinstalled deps.