Open michael-schantin opened 2 years ago
This seems to be due an issue with how Cypress is counting tests and the stats that get passed to the reporter.
Consider your repro case (simplified):
describe('shopMunichmagDe', () => {
it.skip('A skipped test', () => {});
it('A normal test', () => {});
});
which yields these stats (truncated):
{
suites: 1,
tests: 1,
passes: 1,
pending: 1,
}
The same tests when run in plain mocha, yield these stats:
{
suites: 1,
tests: 2,
passes: 1,
pending: 1,
}
Note the difference in the number of tests
.
Interestingly, if I swap the order of your tests such that the skipped one is last, I get these stats:
{
suites: 1,
tests: 2,
passes: 1,
pending: 1,
}
I tested with different numbers of tests in various order and what I found is that Cypress is inconsistently reporting the number of tests
. You're seeing incorrect reporter output because mochawesome uses this tests
stat to calculate other stat values and expects all tests, whether skipped or not, to be included in the count.
Describe the bug In Cypress it is possible to temporarily skip tests by specifying
it.skip
.These tests appear in the report as pending (which, strictly speaking, is not correct, a separate type Skipped would be the better choice here). What is worse, however, is that they affect the number of tests that fail. This number should remain unaffected.
Code Reproduce
Expected behavior
Screenshots
Environment (please complete the following information):
Additional context N/A