search

webdriverio-community / wdio-cucumberjs-json-reporter

A WDIO reporter that creates CucumberJS JSON files for WebdriverIO
https://webdriver.io/docs/wdio-cucumberjs-json-reporter
MIT License
24 stars 32 forks source link

Missing generating some json files after test #42

Open szory opened 3 years ago

szory commented 3 years ago

Hi, I am using: wdio-cucumberjs-json-reporter (2.0.2), webdriverio 6.3.4

and generating a html report with: multiple-cucumber-html-reporter 1.18.0 using the json created.

Problem is with missing generating some json files (not every one), which should be created after finish every test. We have a over 150 tests which are divided into 7 sets. These sets have from 13 - 27 tests. This missing of generating tests is very random, sometimes is missing some tests in set number 2 and some times in set numer 6. Does any one encountered similar problem?

wswebcreation commented 3 years ago

Hi @szory

Thanks for filing an issue, what could happen in this case is that the feature has the same name and that it's overwriting / concatenating the data. Can you first check in all your feature files if the feature names are all unique?

szory commented 3 years ago

Hi, Thx for quick response. I've just checked feature files and all the names are unique....

wswebcreation commented 3 years ago

can you then provide a reproducible example so I can debug it?

szory commented 3 years ago

we run these tests on Jenkins, that would be ok for you if I send Console Output from Jenkins

wswebcreation commented 3 years ago

Sorry @szory

Yes, can you send me the Console Output and also a log of the files that are generated?

szory commented 3 years ago

Hi @wswebcreation Sorry for that huge delay....

this is the piece of Jenkins Console Output. In this sample of log is missing report called: IOOS Online Portal Password Reset:

06:29:45  2021-02-04T05:29:44.843Z INFO @wdio/local-runner: Start worker 0-70 with arg: 
06:29:45  [0-70] 2021-02-04T05:29:45.231Z INFO @wdio/local-runner: Run worker command: run
06:29:45  [0-70] 2021-02-04T05:29:45.377Z INFO webdriverio: Initiate new session using the ./protocol-stub protocol
06:29:45  [0-70] RUNNING in chrome - /src/ui/features/iqos_portal/password-reset.feature
06:29:45  [0-70] 2021-02-04T05:29:45.656Z INFO webdriverio: Initiate new session using the webdriver protocol
06:29:50  [0-70] 2021-02-04T05:29:50.915Z INFO @wdio/browserstack-service: Update job with sessionId a8b233f08c64ee57926fe8b6708d8f95ef152357
06:29:51  [0-70] 2021-02-04T05:29:51.333Z INFO @wdio/browserstack-service: Windows 10 chrome session: https://automate.browserstack.com/builds/7cad3678d9bd81826e6c821092f0edd559aae365/sessions/a8b233f08c64ee57926fe8b6708d8f95ef152357
06:30:01  [0-70] 2021-02-04T05:30:01.040Z INFO @wdio/browserstack-service: Update job with sessionId a8b233f08c64ee57926fe8b6708d8f95ef152357
06:30:01  [0-70] �[32m[INFO ]: Test email created
06:30:02  [0-70] �[32m[INFO ]: User created
06:32:44  [0-70] Error in "IOOS Online Portal Password Reset: Password Reset: And I extract "Reset Password Url" within latest email titled as "Reset Password""
06:32:44  Error: function timed out, ensure the promise resolves within 60000 milliseconds
06:32:44      at Timeout._onTimeout (/home/jenkins/agent/workspace/DCE 2.0 Test Automation Framework/UI/UVT - testing/node_modules/cucumber/lib/user_code_runner.js:76:18)
06:32:44      at listOnTimeout (internal/timers.js:549:17)
06:32:44      at processTimers (internal/timers.js:492:7)
06:32:44  [0-70] 2021-02-04T05:32:39.365Z INFO @wdio/browserstack-service: Update (reloaded) job with sessionId a8b233f08c64ee57926fe8b6708d8f95ef152357, failed
06:32:44  [0-70] 2021-02-04T05:32:39.869Z INFO @wdio/browserstack-service: Update job with sessionId 31f059237d3eb781f80a69ae58726101532651d3
06:32:44  [0-70] 2021-02-04T05:32:40.270Z INFO @wdio/browserstack-service: Windows 10 chrome session: https://automate.browserstack.com/builds/7cad3678d9bd81826e6c821092f0edd559aae365/sessions/31f059237d3eb781f80a69ae58726101532651d3
06:32:44  [0-70] �[32m[INFO ]: Test email created
06:32:44  [0-70] �[32m[INFO ]: User created
06:34:08  [0-70] 2021-02-04T05:34:02.626Z WARN webdriver: Request encountered a stale element - terminating request
06:34:08  [0-70] 2021-02-04T05:34:03.837Z WARN webdriver: Request encountered a stale element - terminating request
06:35:04  [0-70] Error in "IQOS Online Portal Password Reset: Password Reset: And I extract "Reset Password Url" within latest email titled as "Reset Password""
06:35:04  Error: function timed out, ensure the promise resolves within 60000 milliseconds
06:35:04      at Timeout._onTimeout (/home/jenkins/agent/workspace/DCE 2.0 Test Automation Framework/UI/UVT - testing/node_modules/cucumber/lib/user_code_runner.js:76:18)
06:35:04      at listOnTimeout (internal/timers.js:549:17)
06:35:04      at processTimers (internal/timers.js:492:7)
06:35:51  [0-70] 2021-02-04T05:35:47.312Z WARN webdriver: Request failed with status 404 due to Session not started or terminated
06:35:51  [0-70] 2021-02-04T05:35:47.339Z WARN webdriver: Request failed with status 404 due to Session not started or terminated
06:35:51  [0-70] 2021-02-04T05:35:47.364Z WARN webdriver: Request failed with status 404 due to Session not started or terminated
06:35:51  [0-70] 2021-02-04T05:35:47.391Z ERROR webdriver: Request failed with status 404 due to Error: Session not started or terminated
06:35:51  [0-70] 2021-02-04T05:35:47.394Z ERROR @wdio/sync: Error: Session not started or terminated
06:35:51      at processTicksAndRejections (internal/process/task_queues.js:97:5)
06:35:51      at Object.afterScenario (/home/jenkins/agent/workspace/DCE 2.0 Test Automation Framework/UI/UVT - testing/src/core/conf/wdio.template.conf.js:368:13)
06:35:51      at TestCaseRunner.aroundTestStep (/home/jenkins/agent/workspace/DCE 2.0 Test Automation Framework/UI/UVT - testing/node_modules/cucumber/lib/runtime/test_case_runner.js:185:28)
06:35:51      at /home/jenkins/agent/workspace/DCE 2.0 Test Automation Framework/UI/UVT - testing/node_modules/cucumber/lib/runtime/test_case_runner.js:253:7
06:35:51  [0-70] 2021-02-04T05:35:47.396Z INFO @wdio/browserstack-service: Update job with sessionId 31f059237d3eb781f80a69ae58726101532651d3
06:36:01  [0-70] 2021-02-04T05:36:01.593Z WARN webdriver: Request failed with status 404 due to Session not started or terminated
06:36:03  [0-70] 2021-02-04T05:36:02.984Z WARN webdriver: Request failed with status 404 due to Session not started or terminated
06:36:04  [0-70] 2021-02-04T05:36:04.356Z WARN webdriver: Request failed with status 404 due to Session not started or terminated
06:36:05  [0-70] 2021-02-04T05:36:05.735Z ERROR webdriver: Request failed with status 404 due to Error: Session not started or terminated
06:36:05  [0-70] 2021-02-04T05:36:05.736Z ERROR @wdio/local-runner: Failed launching test session: Error: Session not started or terminated
06:36:05      at processTicksAndRejections (internal/process/task_queues.js:97:5)
06:36:05      at Runner.endSession (/home/jenkins/agent/workspace/DCE 2.0 Test Automation Framework/UI/UVT - testing/node_modules/@wdio/local-runner/node_modules/@wdio/runner/build/index.js:277:26)
06:36:05      at Runner.run (/home/jenkins/agent/workspace/DCE 2.0 Test Automation Framework/UI/UVT - testing/node_modules/@wdio/local-runner/node_modules/@wdio/runner/build/index.js:163:18)
06:36:05  [0-70] FAILED in chrome - /src/ui/features/iqos_portal/password-reset.feature

we have also this code in WDIO.config.js:

cucumberOpts: {
    retry: 2
}

retry mechanism failed at some reason that we've got 404 and json report was not generated

Thx in advance for your any help/advise

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

ianhomer commented 2 years ago

I encountered a similar situation (we're wdio / BrowserStack), but the signal looks similar. If BrowserStack is unavailable then we don't get a report.

I created a custom reporter to troubleshoot, e.g.

import WDIOReporter from '@wdio/reporter';
class MyCustomReporter extends WDIOReporter {
    public onRunnerStart(runner) {
        console.log('Runner start');
    }
}

and then register with reporters: ['cucumberjs-json', MyCustomReporter].

With this I see that @wdio/reporter events don't seem to get fired in this situation. For my situation it's not a wdio-cucumberjs-json-reporter issue here, more that the wdio worker is not even calling the reporting if a runner fails to start.