adamgruber / mochawesome

A Gorgeous HTML/CSS Reporter for Mocha.js
https://gitter.im/mochawesome/general
MIT License
1.06k stars 160 forks source link

Filename incorrect when tests fail as other #402

Open ittaibaratz opened 2 months ago

ittaibaratz commented 2 months ago

Describe the bug

I am using the following report name + Cypress 9.4.1 (Yes old):

cypress/reports/mochawesome/mochawesome_[status]_[name]  

In one of my suites, one test fails. The JSON top level stats for some reason bucket it as "others". Note it did not fail in a before hook:

"stats": {
"suites": 4,
"tests": 8,
"passes": 7,
"pending": 1,
"failures": 0,
"start": "2024-06-27T16:24:44.641Z",
"end": "2024-06-27T16:26:48.837Z",
"duration": 124196,
"testsRegistered": 9,
"passPercent": 87.5,
"pendingPercent": 11.11111111111111,
"other": 1,
"hasOther": true,
"skipped": 0,
"hasSkipped": false
}

Failure:

{
"title": "amazon s3 enable saving",
"fullTitle": "Destinations Accounts w/ records amazon s3 enable saving",
"timedOut": null,
"duration": 65368,
"state": "failed",
"speed": null,
"pass": false,
"fail": true,
"pending": false,
"context": "[\n  \"/home/jenkins/workspace/form-platform-ui-client_PR-11259/cypress/screenshots/destinations/destination-accounts-spec.js/<Test name>\",\n  \"/home/jenkins/workspace/form-platform-ui-client_PR-11259/cypress/node_modules/cypress-multi-reporters\"\n]",
"code": "...",
"diff": null
},
"uuid": "00030964-c122-4062-a652-f11e59ae58b2",
"parentUUID": "0a899bf8-80c8-47f5-bf4e-6a00768c20b3",
"isHook": false,
"skipped": false

The result filename is "mochawesome_pass_destination-accounts-spec.json"

Code Reproduce This is a private repo and the issue does not reproduce often since the test is flakey...

Expected behavior My expectation is if other > 0, filename should be fail and not pass.

Screenshots If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

Additional context Add any other context about the problem here.

I can calculate the stats from the JSON, was just hoping to rely on the filename.