Test case is not failing when expect().toEqual() fails in playwright

[arvinder06]

👓 What did you see?

I have a core framework created using playwright and cucumber-js which I am compiling to CommonJs and using the function in a downstream Automation framework. I have all the Assertion define in the core framework which I am using in the application framework. There is a function in my core framework as below: public static async assertEquals(actual: any, expected: any, description: string) { Log.info(Verifying that ${description} is equals to '${expected}'); expect(actual,Expected '${expected}' should be EQUAL to Actual '${actual}').toStrictEqual(expected); }

I am calling this function in the application framework as below: Assert.assertEquals("1","2","Compare 1 and 2");

✅ What did you expect to see?

I am expecting that the step I am calling Assert code should fail, however it is showing the error but the cucumber test scenario is passed.

Logs

` if (!testInfo) return matcher.call(target, ...args); ^ Ze [Error]: expect(received).toStrictEqual(expected) // deep equality

Expected: "2" Received: "1" at Proxy. ( matcherResult: { actual: '1', expected: '2', message: '\x1B[2mexpect(\x1B[22m\x1B[31mreceived\x1B[39m\x1B[2m).\x1B[22mtoStrictEqual\x1B[2m(\x1B[22m\x1B[32mexpected\x1B[39m\x1B[2m) // deep equality\x1B[22m\n' + '\n' + 'Expected: \x1B[32m"2"\x1B[39m\n' + 'Received: \x1B[31m"1"\x1B[39m', name: 'toStrictEqual', pass: false } } 1 scenario (1 passed) 5 steps (5 passed) 1m05.082s (executing steps: 0m30.025s)`

However, when I use the expect("1").toStrictEqual("2"), directly in the application framework, it fails the steps as expected. So the problem I am facing is when I have this code in a core framework and use that framework in the application framework.

[arvinder06]

Moreover, when I am using throw new Error("failed"); in the core framework, it is throwing the error but not failing the cucumber test case.

[davidjgoss]

Cucumber will fail the scenario if an error bubbles up from a step, including assertion errors.

It's hard to know what might be going on from what you've described, but you seem to be mixing frameworks (Playwright Test and Cucumber, which are both test runners in their own right) which is usually a recipe for unexpected results.

In particular, it looks like you're using expect from @playwright/test, which from a quick look at their code would expect to be called in the context of a Playwright Test case.

If you can post a minimum reproducible example here, the community might be able to give some pointers, but I'm pretty sure this isn't going to be a Cucumber issue.

[michael-lloyd-morris]

I'm using Playwright with Cucumber here with the Chai assertion library:

https://github.com/michael-lloyd-morris/portfolio

You do not have to use the Playwright test runner to use Playwright as the browser driver - and to be honest I'd advise against trying. As David pointed out Playwright's assertions are customized to it's runner. Using a general purpose assertion library like Chai should help.

[github-actions[bot]]

This issue is stale because it has been open for 3 weeks with no activity. Remove the stale label or comment or this will be closed in another 5 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.