wdio-html-nice-reporter cannot generate aggregated report when there are a lot of suites and cases

[wei-y]

I am using wdio-html-nice-reporter and wdio 7.7.5 and found that the master report cannot be generated when there are many test suites.

There are 29 test suites, each with its own reports, in my test result and report of each one are generated nicely, but when aggregating, I got an error in log: Html Generation processing ended in error: RangeError: Invalid string length. I located this error generated from this line: https://github.com/rpii/wdio-html-reporter/blob/c9d3ecda5e2d269ac4a604e59e9de5678fd96879/src/htmlGenerator.ts#L191 After googling a bit I found the possible reason could be that the object is too big and out of memory.

I am wondering if it is possible not to generate the JSON file when aggregating, or at least skip the JSON but still generate the HTML in case of error.

package.json

{
  "name": "webdriverio-proto",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "@rpii/wdio-report-events": "^0.1.3",
    "@wdio/cli": "^7.7.5",
    "allure-commandline": "^2.13.8",
    "bent": "^7.3.12",
    "dayjs": "^1.10.6",
    "faker": "^5.5.3",
    "got": "^11.8.2",
    "lodash": "^4.17.21",
    "log4js": "^6.3.0",
    "mssql": "^6.3.1",
    "source-map-support": "^0.5.19",
    "ssh2-promise": "^1.0.0",
    "typescript-string-operations": "^1.4.1",
    "wdio-html-nice-reporter": "^7.7.12",
    "wdio-image-comparison-service": "^2.3.0"
  },
  "devDependencies": {
    "@wdio/allure-reporter": "^7.7.5",
    "@wdio/junit-reporter": "^7.7.5",
    "@wdio/local-runner": "^7.7.5",
    "@wdio/mocha-framework": "^7.7.5",
    "@wdio/spec-reporter": "^7.7.5",
    "@wdio/sync": "^7.7.5",
    "chromedriver": "^91.0.1",
    "wdio-chromedriver-service": "^7.2.0"
  }
}

wdio.config.js (only related part)

const { ReportAggregator } = require("wdio-html-nice-reporter");
const log4js = require("log4js");

exports.config = {
  reporters: [
    "spec",
    [
      "html-nice",
      {
        debug: true,
        outputDir: "./reports/html-reports/",
        filename: "report.html",
        reportTitle: "Test Result",

        //to show the report in a browser when done
        showInBrowser: false,

        //to turn on screenshots after every test
        useOnAfterCommandForScreenshot: false,

        // to use the template override option, can point to your own file in the test project:
        // templateFilename: path.resolve(__dirname, '../template/wdio-html-reporter-alt-template.hbs'),

        // to add custom template functions for your custom template:
        // templateFuncs: {
        //     addOne: (v) => {
        //         return v+1;
        //     },
        // },

        //to initialize the logger
        LOG: log4js.getLogger("default"),
      },
    ],
  ],

  onPrepare: async function (config, capabilities) {
    let reportAggregator = new ReportAggregator({
      outputDir: "./reports/html-reports/",
      filename: "master-report.html",
      reportTitle: "Master Report",
      browserName: capabilities.browserName,
      collapseTests: true,
      showInBrowser: true,
      LOG: console,
      // to use the template override option, can point to your own file in the test project:
      // templateFilename: path.resolve(__dirname, '../template/wdio-html-reporter-alt-template.hbs')
    });
    reportAggregator.clean();
    global.reportAggregator = reportAggregator;
  },

  onComplete: function (exitCode, config, capabilities, results) {
    (async () => {
      await global.reportAggregator.createReport();
    })();
  },
};

[Hayk-S]

With the same WDIO version, I am getting the below error and Html report is not being generated

Report Aggregation started                                                                
Aggregated 1 specs, 1 suites, 1 reports,                                                                                                                                   
Html Generation started                                                                                                                                  
Html Generation processing ended in error: TypeError: Cannot read property 'replace' of undefined
Report Aggregation completed

[rpii]

I have not yet tested with 7.7.5. I will do that soon. you can turn off saving the .json file, but data is aggregated in memory to build the composite report. So it will use memory

Rich

On Thu, Jul 15, 2021 at 1:16 PM Hayk @.***> wrote:

With the same WDIO version, I am getting the below error and Html report is not being generated

Report Aggregation started Aggregated 1 specs, 1 suites, 1 reports, Html Generation started Html Generation processing ended in error: TypeError: Cannot read property 'replace' of undefined Report Aggregation completed

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/rpii/wdio-html-reporter/issues/64#issuecomment-880984138, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEABIEO2TLDMYW6IARKJVNDTX46V5ANCNFSM5AMWXCFA .

[wei-y]

I have not yet tested with 7.7.5. I will do that soon. you can turn off saving the .json file, but data is aggregated in memory to build the composite report. So it will use memory Rich … On Thu, Jul 15, 2021 at 1:16 PM Hayk @.***> wrote: With the same WDIO version, I am getting the below error and Html report is not being generated Report Aggregation started Aggregated 1 specs, 1 suites, 1 reports, Html Generation started Html Generation processing ended in error: TypeError: Cannot read property 'replace' of undefined Report Aggregation completed — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#64 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEABIEO2TLDMYW6IARKJVNDTX46V5ANCNFSM5AMWXCFA .

I tried to generate the aggregated report commenting out the line in question. The aggregated report is generated without issue. Perhaps it is something only in the JSON.stringify(). The same was happening with the deprecated html-reporter.

[rpii]

Ok. so I added an exception handler around the json.stringify. Should run to completion now..

[wei-y]

@rpii Could you please let me know where I can see this change? I'm using npm to manage the package but it looks like it's not updated for a while at npmjs.org. Also cannot see the change in the master branch

[rpii]

version 7.7.14 and the change are now pushed to gitlab

On Sun, Jul 18, 2021 at 6:54 PM wei-y @.***> wrote:

@rpii https://github.com/rpii Could you please let me know where I can see this change? I'm using npm to manage the package but it looks like it's not updated for a while at npmjs.org. Also cannot see the change in the master branch

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/rpii/wdio-html-reporter/issues/64#issuecomment-882170406, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEABIEN7652KP2RGM5HKYHTTYOAT7ANCNFSM5AMWXCFA .

[rpii]

I will have to look into this. to be honest Im not very interested in cucumber. Use the old version in the meanwhile.

Rich

On Wed, Jul 14, 2021 at 9:10 PM wei-y @.***> wrote:

I am using wdio-html-nice-reporter and wdio 7.7.5 and found that the master report cannot be generated when there are many test suites.

There are 29 test suites in my test result and report of each one are generated nicely, but when aggregating, I got an error in log: Html Generation processing ended in error: RangeError: Invalid string length. I located this error generated from this line: https://github.com/rpii/wdio-html-reporter/blob/c9d3ecda5e2d269ac4a604e59e9de5678fd96879/src/htmlGenerator.ts#L191 After googling a bit I found the possible reason could be that the object is too big and out of memory.

I am wondering if it is possible not to generate the JSON file when aggregating, or at least skip the JSON but still generate the HTML in case of error.

package.json

{ "name": "webdriverio-proto", "version": "1.0.0", "description": "", "main": "index.js", "scripts": { "test": "echo \"Error: no test specified\" && exit 1" }, "keywords": [], "author": "", "license": "ISC", "dependencies": { @./wdio-report-events": "^0.1.3", @./cli": "^7.7.5", "allure-commandline": "^2.13.8", "bent": "^7.3.12", "dayjs": "^1.10.6", "faker": "^5.5.3", "got": "^11.8.2", "lodash": "^4.17.21", "log4js": "^6.3.0", "mssql": "^6.3.1", "source-map-support": "^0.5.19", "ssh2-promise": "^1.0.0", "typescript-string-operations": "^1.4.1", "wdio-html-nice-reporter": "^7.7.12", "wdio-image-comparison-service": "^2.3.0" }, "devDependencies": { @./allure-reporter": "^7.7.5", @./junit-reporter": "^7.7.5", @./local-runner": "^7.7.5", @./mocha-framework": "^7.7.5", @./spec-reporter": "^7.7.5", @./sync": "^7.7.5", "chromedriver": "^91.0.1", "wdio-chromedriver-service": "^7.2.0" } }

wdio.config.js (only related part)

const { ReportAggregator } = require("wdio-html-nice-reporter");const log4js = require("log4js"); exports.config = { reporters: [ "spec", [ "html-nice", { debug: true, outputDir: "./reports/html-reports/", filename: "report.html", reportTitle: "Test Result",

    //to show the report in a browser when done
    showInBrowser: false,

    //to turn on screenshots after every test
    useOnAfterCommandForScreenshot: false,

    // to use the template override option, can point to your own file in the test project:
    // templateFilename: path.resolve(__dirname, '../template/wdio-html-reporter-alt-template.hbs'),

    // to add custom template functions for your custom template:
    // templateFuncs: {
    //     addOne: (v) => {
    //         return v+1;
    //     },
    // },

    //to initialize the logger
    LOG: log4js.getLogger("default"),
  },
],

],

onPrepare: async function (config, capabilities) { let reportAggregator = new ReportAggregator({ outputDir: "./reports/html-reports/", filename: "master-report.html", reportTitle: "Master Report", browserName: capabilities.browserName, collapseTests: true, showInBrowser: true, LOG: console, // to use the template override option, can point to your own file in the test project: // templateFilename: path.resolve(__dirname, '../template/wdio-html-reporter-alt-template.hbs') }); reportAggregator.clean(); global.reportAggregator = reportAggregator; },

onComplete: function (exitCode, config, capabilities, results) { (async () => { await global.reportAggregator.createReport(); })(); },};

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/rpii/wdio-html-reporter/issues/64, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEABIEJT5QCMGZHS2LEOJZLTXZNP3ANCNFSM5AMWXCFA .

[borceacristiann]

Hi, I have same issue, when the spec is too long is thrown RangeError: Invalid string length and the report is not generated. Are there any updates on this issue?

[GPSC1]

@wei-y I hv a question related to nice reported output. Do you also see test steps under "Test passed" instead of Test scanarios?.. Example- If there are 2 scenarios and each contains 3 test steps then final report shows "Test passed" as 6 but i needed to keep the count as 2 i.e. Test scenarios count. Attached image