How can I get consolidated json report, in protractor-cucumber framework with serenity reports

[kumarvarun1252]

Hi,

I am trying to get a consolidate JSON report for my protractor-cucumber framework with serenity reports and node-js. I see there is a serenity-gradle-plugin, how can I use it in my framework?

Can I add serenity-gradle-plugin as a dependency in my package.json? My package.json looks like this

  "name": "UI-protractor-tests",
  "version": "1.0.0",
  "description": "Serenity reports with classic Protractor and Cucumber setup",
  "main": "index.js",
  "scripts": {
    "preprotractor": "webdriver-manager update --standalone --versions.chrome=2.32",
    "protractor": "protractor ./protractor.conf.js",
    "test": "npm run protractor --silent",
    "pretest": "serenity update",
    "report": "serenity run"
  },
  "repository": {
    "type": "git",
    "url": "https://github.com/kumarvarun1252/Hygieia"
  },
  "keywords": [
    "serenity-js",
    "protractor",
    "cucumber"
  ],
  "author": "",
  "license": "Apache-2.0",
  "devDependencies": {
    "chai": "4.0.2",
    "chai-as-promised": "7.0.0",
    "chai-smoothie": "^0.2.0",
    "cucumber": "1.3.2",
    "loglevel": "1.5.0",
    "npm-failsafe": "^0.1.0",
    "protractor": "5.2.0",
    "protractor-cucumber-framework": "3.1.2",
    "rimraf": "2.5.4",
    "serenity-cli": "^0.5.0",
    "serenity-js": ">= 1.7.0"
  },
  "dependencies": {
    "serenity-js": "^1.8.0",
    "ssl-root-cas": "^1.2.4"
  }
}

[jan-molak]

Please have a look at this section of the docs.

serenity-cli is a wrapper around Serenity BDD jar, so you won't need the serenity-gradle-plugin if you use it.

Let me know if this helps! Jan

[BlacktoBlue]

If I understand this correctly, then I should be able to combine reports from Serenity-BDD projects as well? If this is the case how do I go about doing that? I have a Maven project which runs serenity using the Serenity Maven Plugin (latest version) and I have a Serenity-JS version. each of these can generate their reports separately but i am unsure as how to combine them?

What I am ultimately expecting to see is one test report with all tests tun (I have tagged the website as @application:Website and the service as @application:Service).

I have tried the following: Generating .json files only and combining into /target/site/serenity and then running npm serenity run. Generating full reports for both and combining into the /target/site/serenity and then running npm serenity run.

please can you give me some direction, if this is even possible?

Serenity versions in maven are all latest (1.7.0 plugin is 1.6.9) npm Serenity v 5.4.3

[jan-molak]

@BlacktoBlue - the reporting process works like this:

• Serenity/JS generates the .json reports and screenshots
• Serenity BDD analyses the .json reports but also scans the directory structure of where your feature files are stored in order to understand the names of the capabilities (the sub-directories in which the .feature files live). That means that you need to have access to the original feature files when the .html report is generated

What's your project structure like? Do the JS and Java projects live in the same repository or separate one? Where do the feature files live?

J

[BlacktoBlue]

@jan-molak currently the Serenity/JS tests sit within their own functional test repository, and the Serenity BDD (For services) lives within the project repository and the feature files live within those repos.

My ultimate Aim is to be able to have several pipelines for different products, which all have their own Serenity report, but they also generate a single report for all products.

[kumarvarun1252]

@jan-molak right now, if I have 4 scenarios, I am seeing 4 separate json files. I am looking for one consolidated json file with all the executed scenarios. Is it possible? I see the consolidated html report, but to integrate my project in our CI/CD process, I will have to provide them a consolidated json report file which includes the test execution report of all the scenarios. Thanks.

[JeffGaly]

You found any solution to consolidate all json file ?

[jan-molak]

Hey @kumarvarun1252, @BlacktoBlue, @JeffGaly! Apologies for taking long to respond, but I was focusing on Serenity/JS 2.0.

@kumarvarun1252 - to integrate my project in our CI/CD process, I will have to provide them a consolidated json report file which includes the test execution report of all the scenarios. Thanks.

I'd like to better understand the use case for the consolidated JSON report. Could you please elaborate a bit further on the format of the report your CI/CD system requires?

My ultimate Aim is to be able to have several pipelines for different products, which all have their own Serenity report, but they also generate a single report for all products.

@BlacktoBlue - that's doable, but you'd need to introduce some additional processing steps to your pipeline. Since Serenity BDD requires both the .feature files and the .json reports, you'd need some sort of an aggregation step that copies (merges) both the .feature files and .json test reports across your products into a single directory structure and then generate the HTML report in there.

[bijuvateswarath]

Hi @kumarvarun1252, @BlacktoBlue, @JeffGaly - I am trying to solve a similar problem with multiple java maven modules generating their own reports correctly . I want parent module ( or a separate project ) consolidate the reports . Is that possible ? Or if there is a workaround I can do.

[viper3400]

Hi, this one is quite old. Has someone a solution for this?

As far as I know, there is no build-in way to generate consolidated json report file with Serenity/JS. What I do as workaround is to configure the Artifact Archiver to store all artifacts into a separate 'results' directory.

Then I pass this directory as source to the SerenityBDD cli. I let the client generate the Serenity BDD report into a different destination directory, as well.

The original json results files stay untouched. Given the fact, we are using javascript or typescript, I wrote my own script which will parse the all the *.json result files and convert them into whatever format is necessary.

Though, I do not have the need to combine the json files on my own. But I could imagine, that there are more solutions like this out there. You then can simply call your custom js/ts script via npm scripts after you're tests have been finished.