YOU54F / cypress-multi-reporters

A maintained fork of mocha-multi-reporters compatible with mocha 6+ and below
MIT License
74 stars 6 forks source link

Provide generic cross-reporter substitution mechanism for dynamic file name output #61

Closed brettz9 closed 4 years ago

brettz9 commented 4 years ago

Expected behavior

The mocha-junit-reporter reporter has a nice feature to allow dynamic substitution of a portion of the file name (if "[hash]" is found within mochaFile, it will be auto-substituted with a unique hash). Cypress docs also provide an example of its use: https://docs.cypress.io/guides/tooling/reporters.html#Merging-reports-across-spec-files

In the context of multi-reporters, Cypress has an example of this also (again with mocha-junit-reporter): https://docs.cypress.io/guides/tooling/reporters.html#Multiple-reporters.

I would like to see a more generic mechanism that:

  1. Works across different reporters, even if those reporters didn't have built-in support for such within-file-name substitutions. For example, I am using https://github.com/mocha-community/json-file-reporter/ to build JSON files with Mocha's built in simple JSON reporter (no need for mocha-junit-reporter or mochawesome), but this reporter does not itself allow substitutions within file names.
  2. Does not limit substitutions within such (file name) properties to hashes, but allows a reporter option to pass in a literal string. For example, in https://github.com/indexeddbshim/indexeddbshim , I am using this approach with a fork of mocha-multi-reporters to add a test group type to the resulting file names, so that the multiple generated file names can be more human readable, e.g., building file-mock.json, file-fake.json, etc., depending on the test type (having human readable names can still be useful to get at even if one is typically building separate files with the intent to eventually merge them). Even if one can pass in the whole file name reporter option such that cypress-multi-reporters can pick it up, this would be less than ideal in that it can be nice to store the longer base path in the config file, and just specify the test file group in the command line call. So in indexeddbshim, I only need my CLI call to be mocha --reporterOptions configFile=mocha-multi-reporters.json,mmr-output=-mock which gets applied to the MMR config file (JSON below) whereby my MMR fork substitutes {num} (which I probably should have named {id} or possibly repurposed "[hash]" from the junit reporter) with the mmr-output CLI argument (i.e., here "mock") to dynamically substitute the output value (if implementing, we might want a config option to indicate which reporters and which of their properties should get dynamic substitutions); there is no need to respecify test-support/results/file... on the command line:
{
  "mochajsJsonFileReporterReporterOptions": {
     "output": "test-support/results/file{num}.json"
  }
}

(Perhaps a config could also be set to auto-set a GUID if that were desired by other projects.)

Actual behavior

I am otherwise not able to build multiple Mocha result files, unless using the junit reporter or avoiding taking advantage of multiple reporters at the same time.

While I could file this as an issue to https://github.com/mocha-community/json-file-reporter/ and hope they implement, I would think this may be a common enough need, especially when having a complex enough set-up to want multiple reporters, to make a generic mechanism.

Thanks!

YOU54F commented 4 years ago

Hey, sounds like a great idea, I use that very junit reporters hashing function for my xml results, but have a little function to generate unique report names, which is separate to this package.

I don't have the time to fully digest the details atm, but will look into it in a bit more detail over the weekend. Happy to get others input as well 👍

It is nice to see this repo getting some use, as I can't take credit for it, that goes to the original author @stanleyhlng but it is nice to carry it on, and to make it a bit better along the way!

YOU54F commented 4 years ago

@brettz9 should be getting released in a few minutes as 1.3.0

brettz9 commented 4 years ago

Super, thanks a lot!