[BUG] Playwright config screenshot (only-on-failure) - no screenshot is saved when test failed

[elainefy]

Context:

• Playwright Version: [1.23.4]
• Operating System: [Windows 10 pro]
• Node.js version: [16.16.0]
• Browser: [Chromium]
• Extra: [Using Typescript]

Code Snippet

playwright.config.ts

import { PlaywrightTestConfig } from '@playwright/test';

const config: PlaywrightTestConfig = {
  testDir: './tests',
  workers: 2,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  outputDir: './test-results',
  reporter: 'html',
  timeout: 60000,
  expect: { timeout: 10000 },
  use: {
    headless: false,
    actionTimeout: 10000,
    trace: 'on',
    screenshot: 'only-on-failure',
    ignoreHTTPSErrors: true,
  }
};

export default config;

Describe the bug

When I set the screenshot config to 'only-on-failure', no screenshot is captured in test-results folder or displayed in html report.

[dgozman]

@elainefy This works for on my computer. Could you please share a repro that we can run locally?

[elainefy]

Hi @dgozman, please refer to the failed test demo at https://github.com/elainefy/playwright_screenshot. Command to execute the test -yarn pw

No screenshot found in the test report.

Test run log:

`Running 1 test using 1 worker
  1) testing.test.ts:16:5 › Feature: Testing › Failed test screenshot testing ======================

    Error: expect(received).toBeVisible()
    Call log:
      - expect.toBeVisible with timeout 10000ms
      - waiting for selector "div#test"

      17 |
      18 |         await page.goto('https://www.google.com/');
    > 19 |         await expect(page.locator('div#test')).toBeVisible();
         |                                                ^
      20 |     })
      21 |
      22 | });

        at C:\Users\fyee\Documents\GitHub\playwright_screenshot\tests\testing.test.ts:19:48

    attachment #1: trace (application/zip) ---------------------------------------------------------
    test-results\testing-Feature-Testing-Failed-test-screenshot-testing\trace.zip
    Usage:

        npx playwright show-trace test-results\testing-Feature-Testing-Failed-test-screenshot-testing\trace.zip

    ------------------------------------------------------------------------------------------------

  Slow test file: testing.test.ts (17s)
  Consider splitting slow test files to speed up parallel execution

  1 failed
    testing.test.ts:16:5 › Feature: Testing › Failed test screenshot testing =======================`

[dgozman]

@elainefy Thank you for the repro. This issue is similar to #14854, and should be already fixed in v1.24 release.

[elainefy]

@dgozman Awesome! Thank you for the fixes :)

[ianNoahs]

@dgozman Is their any way to take screenshot if any test cases failed in GitHub Action?