[BUG] Video is not attached on first retrty

[rkhomi]

System info

• Playwright Version: [v1.40.0]
• Operating System: [ macOS 13.2]
• Browser: [All,]
• Other info:

Source code

I have overrided the page fixture to check there are no console messages in afterEach test During the test, it catches the error and fails the test in afterEach hook The problem is it does not attach the video on the first retry, only traces are available, the reason the video is needed is that I need to know how to reproduce the console error

export const test = base.extend<Fixtures & CustomConfigOptions>({
    page: async ({ page }, use) => {
        const errors = [];
        page.on('console', message => {
            if (
                message.type() === 'error' &&
                !excludedMessages.some(excludedMessage => message.text().includes(excludedMessage))
            ) {
                errors.push(message);
            }
        });

        await use(page);
        expect(errors).toEqual([]);
    },

});

Config

export default defineConfig<CustomConfigOptions>({
    testDir: './src/specs',
    testMatch: '**/*.spec.ts',
    expect: { timeout: 30000 },
    /* Retry on CI only */
    retries: process.env.APP_ENV === 'pipeline' ? Number(process.env.RETRIES) : 0,
    /* Opt out of parallel tests on CI. */
    workers: Number(process.env.WORKERS),
    /* Maximum time one test can run for. */
    timeout: 8 * 60 * 1000, //this is 8 minutes
    reporter: process.env.APP_ENV === 'pipeline' ? 'blob' : 'html',
    /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
    use: {
        actionTimeout: 30000,
        baseURL: `https://home.${process.env.HOST}`,
        ignoreHTTPSErrors: true,
        bypassCSP: true,
        navigationTimeout: 60000,
        /* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
        trace: process.env.APP_ENV === 'pipeline' ? 'on-first-retry' : 'retain-on-failure',
        screenshot: 'only-on-failure',
        video: 'on-first-retry',
    },

    /* Configure projects for major browsers */
    projects: [
        {
            name: 'local',
            use: {
                mailUrl: `https://mail.${process.env.HOST}`,
                actionTimeout: 30000,
                baseURL: `https://home.${localBaseUrls[organization]}`,
            },
        },

    ],

});

Steps

• Turn on the video in config
• Make it so the application throws a console error
• Run the test so that it caches the console error
• Retry the test

Expected

Video should be available, on the retry when after each hook fails Ideally add the option to fail the test immediately on the console error, because throwing an error in page.on() callback also breaks the UI runner when the error is at the end of the test

Actual

No video is attached at the end of the test after retry on the console error

[pavelfeldman]

A side question while I'm looking at it: trace is a strict superset of the vide, why do you need both?

[rkhomi]

A side question while I'm looking at it: trace is a strict superset of the vide, why do you need both?

traces are also unavailable even without retry when console error occur, I am unable to debug it

config

trace: 'on',
        screenshot: 'only-on-failure',
        video: 'on',

yarn playwright test --project=local specs/transactions/online-payments/check-pay-online-buttons/stripe-merchant/check-pay-online-stripe-merchant-tt.spec.ts

projects: [
        {
            name: 'local',
            use: {
                mailUrl: `https://mail.${process.env.HOST}`,
                actionTimeout: 30000,
                baseURL: `https://home.${localBaseUrls[organization]}`,
            },
        },
        ]

[pavelfeldman]

Did you select the first retry in the report? Both video and trace are there for me when I run your app with APP_ENV=pipeline

[pavelfeldman]

Closing as per above. Please file issue with the additional details if it does address your issue.