search

ShuiRuTian / playwright-trx-reporter

playwright reporter for trx file format.
7 stars 3 forks source link

Result summary is marked as 'Failed' when there are skipped test cases #12

Open liuqidake opened 5 days ago

liuqidake commented 5 days ago

We recently noticed an issue that the reporter set result summary outcome as 'Failed' when there are skipped test cases and all other test cases passed. Right now it looks like as long as the total test cases count does not match passed test cases counts, the result summary will be set to 'Failed' regardlessly. An option should be provided to allow callers to customize the result, something like 'treatSkippedAsPassed' to make sure the test result is marked as passed if there are only passed and skipped test cases.

ShuiRuTian commented 22 hours ago

pretty reasonable, I think it's even the better default behavior

ShuiRuTian commented 22 hours ago

If possible, could you share a minimal repo to reproduce the issue? Then I could confirm it's fixed

liuqidake commented 4 hours ago

@ShuiRuTian thanks for the reponse. Here is the link of the small repo that you can repro the issue https://github.com/liuqidake/playwright-trx-reporter-bug-repo. There are two test cases and one of them is skipped. After running npx playwright test, you can see ResultSummary's outcome is 'Failed' in result.trx file.