Skipped tests cause the number of failed tests to be decremented by 1.

[michael-schantin]

Describe the bug In Cypress it is possible to temporarily skip tests by specifying it.skip.

These tests appear in the report as pending (which, strictly speaking, is not correct, a separate type Skipped would be the better choice here). What is worse, however, is that they affect the number of tests that fail. This number should remain unaffected.

Code Reproduce

• shopMunichmagDe

Expected behavior

• Skipped tests should not affect the number of tests that failed.

Screenshots

• 

Environment (please complete the following information):

• macOS 12.1
• mochawesome 7.0.1
• cypress 9.2.0
• Node 16.13.1

Additional context N/A

[adamgruber]

This seems to be due an issue with how Cypress is counting tests and the stats that get passed to the reporter.

Consider your repro case (simplified):

describe('shopMunichmagDe', () => {
  it.skip('A skipped test', () => {});
  it('A normal test', () => {});
});

which yields these stats (truncated):

{
  suites: 1,
  tests: 1,
  passes: 1,
  pending: 1,
}

The same tests when run in plain mocha, yield these stats:

{
  suites: 1,
  tests: 2,
  passes: 1,
  pending: 1,
}

Note the difference in the number of tests.

Interestingly, if I swap the order of your tests such that the skipped one is last, I get these stats:

{
  suites: 1,
  tests: 2,
  passes: 1,
  pending: 1,
}

I tested with different numbers of tests in various order and what I found is that Cypress is inconsistently reporting the number of tests. You're seeing incorrect reporter output because mochawesome uses this tests stat to calculate other stat values and expects all tests, whether skipped or not, to be included in the count.