'failures' in mochawesome-report is not accurate.

[promise-1999]

Current behavior

I have three cypress test cases, one executed successfully, one marked as skip, and the other constructed an exception. The following is the result of cypress.

But in the mochawesome-report, failures is 0.

Desired behavior

No response

Test code to reproduce

test.cy.js

describe( 'test', () =>
 {

    it( 'test1', () => {

    } )

    it.skip( 'test2', () => {

    } )

    it( 'test3', () => {
       throw new Error('This test case should fail');
    } )
 } )

Cypress Version

13.11.0

Node version

v18.20.3

Operating System

Ubuntu 20.04

Debug Logs

No response

Other

No response

[MikeMcC399]

@promise-1999

The Node.js and Ubuntu versions you are using are no longer supported. The version of Cypress is also old.

You need to be using a minimum Node.js version 18.x and Ubuntu 20.x (see System Requirements). If you believe you have found a bug in Cypress that you want to report, then you should make sure that it is reproducible on Cypress 13.x - preferably the latest version.

You can also ask for help from the Cypress technical community on Discord

(click on button)

[jennifer-shehane]

Closing. We can reopen if an example is provided in our most recent version of Cypress demonstrating the issue.

[promise-1999]

@MikeMcC399 I switched to testing on the required system, and I'm sure the problem still exists.

[MikeMcC399]

@promise-1999

I followed the documentation in

https://docs.cypress.io/guides/tooling/reporters#Cypress-configuration-5 using:

• Cypress 13.11.0
• mochawesome@7.1.3

and I was able to reproduce your issue.

It seems if the order of the tests is changed, then the reported results are correct. So it's a bug, but you may be able to work around it.

spec.cy.js

describe('test', () => {
  it('test3', () => {
    throw new Error('This test case should fail');
  })
  it('test1', () => { })
  it.skip('test2', () => { })
})

produces

{
  "stats": {
    "suites": 1,
    "tests": 3,
    "passes": 1,
    "pending": 1,
    "failures": 1,
    "start": "2024-06-14T11:19:52.429Z",
    "end": "2024-06-14T11:19:52.829Z",
    "duration": 400,
    "testsRegistered": 3,
    "passPercent": 50,
    "pendingPercent": 33.33333333333333,
    "other": 0,
    "hasOther": false,
    "skipped": 0,
    "hasSkipped": false
  },
  "results": [
    {
      "uuid": "dc102c24-ebed-4c0d-a3d4-cb86a7dfb7f4",
      "title": "",
      "fullFile": "cypress\\e2e\\spec.cy.js",
      "file": "cypress\\e2e\\spec.cy.js",
      "beforeHooks": [],
      "afterHooks": [],
      "tests": [],
      "suites": [
        {
          "uuid": "55d412a9-6995-412d-8e4d-f8c977172bde",
          "title": "test",
          "fullFile": "",
          "file": "",
          "beforeHooks": [],
          "afterHooks": [],
          "tests": [
            {
              "title": "test3",
              "fullTitle": "test test3",
              "timedOut": null,
              "duration": 302,
              "state": "failed",
              "speed": null,
              "pass": false,
              "fail": true,
              "pending": false,
              "context": null,
              "code": "throw new Error('This test case should fail');",
              "err": {
                "message": "Error: This test case should fail",
                "estack": "Error: This test case should fail\n    at Context.eval (webpack://cy-mochawesome/./cypress/e2e/spec.cy.js:3:10)",
                "diff": null
              },
              "uuid": "32b09751-35ad-442f-85a1-0e067af5d120",
              "parentUUID": "55d412a9-6995-412d-8e4d-f8c977172bde",
              "isHook": false,
              "skipped": false
            },
            {
              "title": "test1",
              "fullTitle": "test test1",
              "timedOut": null,
              "duration": 11,
              "state": "passed",
              "speed": "fast",
              "pass": true,
              "fail": false,
              "pending": false,
              "context": null,
              "code": "",
              "err": {},
              "uuid": "9e195bf3-a980-4ecd-ad46-2aebd63698b1",
              "parentUUID": "55d412a9-6995-412d-8e4d-f8c977172bde",
              "isHook": false,
              "skipped": false
            },
            {
              "title": "test2",
              "fullTitle": "test test2",
              "timedOut": null,
              "duration": 0,
              "state": "pending",
              "speed": null,
              "pass": false,
              "fail": false,
              "pending": true,
              "context": null,
              "code": "",
              "err": {},
              "uuid": "1a6eae37-463f-4d35-88a4-6146e6b938cb",
              "parentUUID": "55d412a9-6995-412d-8e4d-f8c977172bde",
              "isHook": false,
              "skipped": false
            }
          ],
          "suites": [],
          "passes": [
            "9e195bf3-a980-4ecd-ad46-2aebd63698b1"
          ],
          "failures": [
            "32b09751-35ad-442f-85a1-0e067af5d120"
          ],
          "pending": [
            "1a6eae37-463f-4d35-88a4-6146e6b938cb"
          ],
          "skipped": [],
          "duration": 313,
          "root": false,
          "rootEmpty": false,
          "_timeout": 2000
        }
      ],
      "passes": [],
      "failures": [],
      "pending": [],
      "skipped": [],
      "duration": 0,
      "root": true,
      "rootEmpty": true,
      "_timeout": 2000
    }
  ],
  "meta": {
    "mocha": {
      "version": "7.0.1"
    },
    "mochawesome": {
      "options": {
        "quiet": false,
        "reportFilename": "mochawesome",
        "saveHtml": false,
        "saveJson": true,
        "consoleReporter": "spec",
        "useInlineDiffs": false,
        "code": true
      },
      "version": "7.1.3"
    },
    "marge": {
      "options": {
        "reportDir": "cypress/results",
        "overwrite": false,
        "html": false,
        "json": true
      },
      "version": "6.2.0"
    }
  }
}

[promise-1999]

@MikeMcC399 Yes, at present, I can avoid this bug by putting the test marked skip at the end.

But you don't need to pay attention to the order, it is definitely better.