DevExpress / testcafe

A Node.js tool to automate end-to-end web testing.
https://testcafe.io
MIT License
9.82k stars 671 forks source link

Screenshots with the takeOnFails option are taken during the test #8244

Closed JakubMaz closed 2 months ago

JakubMaz commented 2 months ago

What is your Scenario?

I want to use an assertion in a try catch block to deal with refreshing some elements on my page. Or refresh the page if something doesn't change.

What is the Current behavior?

Screenshots with the takeOnFails option, are taken during the test. To be more precise, a screen shot is taken when an assertion in a try catch block fails. It's not taking on the actual cause of the error.

What is the Expected behavior?

A screenshot is taken when the actual error occurs which marks the test as 'failed'

What is the public URL of the test page? (attach your complete example)

https://devexpress.github.io/testcafe/example/

What is your TestCafe test code?

import {  Selector, t } from "testcafe";

const devNameInput = Selector('#developer-name');
const loader = Selector('#not-real-selector'); // selector for test

const refreshPage = async () => {
    try {
        await t
            .expect(loader.exists).ok()
            .expect(loader.exists).notOk()
    } catch (error) {
        //empty    
    }}

fixture('Getting Started')

test('My 1st test', async ()  => {
    await t.debug()

    // something that can appear with a delay and recharge the data, 
    // if it does not appear it means that everything has been loaded
    await refreshPage()

    // enters the text to see it on the screenshot
    await t.typeText(devNameInput, 'John Doe', { paste: true });

    await t.expect(1).eql(2);// the actual cause of the test failure
});

Your complete configuration file

module.exports = {
    src: "./tests",
    browsers: ["chrome:--disable-search-engine-choice-screen"],
    baseUrl: "https://devexpress.github.io/testcafe/example/",
    skipJsErrors: true,
    screenshots: {
        "takeOnFails": true
    }
}

Your complete test report

No response

Screenshots

No response

Steps to Reproduce

Add assertion what can faild in try catch block. Run test with screen shot on fail option

TestCafe version

3.6.0

Node.js version

No response

Command-line arguments

testcafe

Browser name(s) and version(s)

No response

Platform(s) and version(s)

No response

Other

No response

PavelMor25 commented 2 months ago

Hello @JakubMaz,

Using try-catch in tests is considered a bad practice. It interferes with TestCafe's built-in error handling and reporting mechanisms, which can lead to the issues you're encountering.

Screenshots taken with the takeOnFails option apply to all potential errors that occur during TestCafe's operation and result in test failures. Handling errors within the test itself, rather than relying on TestCafe's mechanisms, prevents additional screenshots from being captured when an error occurs.

We recommend avoiding the use of try-catch in your tests. Instead, utilize the tools and practices provided by TestCafe for error handling and reporting.