search

reportportal / agent-js-cucumber

Agent to integrate CucumberJS with ReportPortal.
https://www.npmjs.com/package/@reportportal/agent-js-cucumber
Apache License 2.0
11 stars 17 forks source link

Report portal status interrupted for second scenario #118

Closed swathisnice closed 1 year ago

swathisnice commented 1 year ago

Team, RP is working fine with Cucumber Js 6. we upgraded the Cucumber version to 8.8.0 and RP version as 5.1.0(latest). We are seeing issues with Suites having multiple scenarios. RP is creating the launch for the test but executing only one scenario. Later it is showing as ‘Interrupted’ status for the next scenario and stopped. Not executing all the scenarios. We checked with different suites with 1 scenario, it is passing. How to resolve this. Which version of Cucumber >6 supports RP with multiple scenarios in a feature. @chivekrodis @AmsterGet Please help us with a resolution

AmsterGet commented 1 year ago

Hello @swathisnice ! Multiple scenarios in a feature should work with agent 5.1.0 (latest version). Seems like it is a bug in the latest version. Could you provide more details to help us investigate and fix this issue? Did you have any runtime errors?

swathisnice commented 1 year ago

Hi @AmsterGet ,

image

To cross check, we merged all scenarios to one, it is passed and report is generated properly I'm seeing the below error for second scenario after first step

image

(node:1388) UnhandledPromiseRejectionWarning: Error: Item "bi4rj9d12klbndswus" not found at RPClient.startTestItem (@ourapp\node_modules\@reportportal\client-javascript\lib\report-portal-client.js:423:51) at CucumberReportPortalFormatter.onTestStepStartedEvent (@ourapp\node_modules\@reportportal\agent-js-cucumber\modules\api\current.js:274:44) at EventEmitter. (@ourapp\node_modules\@reportportal\agent-js-cucumber\modules\api\current.js:52:23)
at EventEmitter.emit (events.js:412:35) at TestCaseRunner.aroundTestStep (@ourapp\node_modules\@cucumber\cucumber\src\runtime\test_case_runner.ts:165:27) at TestCaseRunner.runAttempt (@ourapp\node_modules\@cucumber\cucumber\src\runtime\test_case_runner.ts:215:18)
at TestCaseRunner.run (@ourapp\node_modules\@cucumber\cucumber\src\runtime\test_case_runner.ts:185:29) at Runtime.runTestCase (@ourapp\node_modules\@cucumber\cucumber\src\runtime\index.ts:89:20) at Runtime.start (@ourapp\node_modules\@cucumber\cucumber\src\runtime\index.ts:116:7) at runCucumber (@ourapp\node_modules\@cucumber\cucumber\src\api\run_cucumber.ts:125:19) at Cli.run (@ourapp\node_modules\@cucumber\cucumber\src\cli\index.ts:78:25) at Object.run [as default] (@ourapp\node_modules\@cucumber\cucumber\src\cli\run.ts:38:14) (Use node --trace-warnings ... to show where the warning was created) (node:1388) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1) (node:1388) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code. (node:1388) UnhandledPromiseRejectionWarning: Error: Launch "bi4rj9d12klbndo8fp" is already finished, you can not add an item to it at RPClient.startTestItem (@ourapp\node_modules\@reportportal\client-javascript\lib\report-portal-client.js:410:19) at CucumberReportPortalFormatter.onTestStepStartedEvent (@ourapp\node_modules\@reportportal\agent-js-cucumber\modules\api\current.js:274:44) at EventEmitter. (@ourapp\node_modules\@reportportal\agent-js-cucumber\modules\api\current.js:52:23)
at EventEmitter.emit (events.js:412:35) at TestCaseRunner.aroundTestStep (@ourapp\node_modules\@cucumber\cucumber\src\runtime\test_case_runner.ts:165:27) at TestCaseRunner.runAttempt (@ourapp\node_modules\@cucumber\cucumber\src\runtime\test_case_runner.ts:215:18)
at TestCaseRunner.run (@ourapp\node_modules\@cucumber\cucumber\src\runtime\test_case_runner.ts:185:29) at Runtime.runTestCase (@ourapp\node_modules\@cucumber\cucumber\src\runtime\index.ts:89:20) at Runtime.start (@ourapp\node_modules\@cucumber\cucumber\src\runtime\index.ts:116:7) at runCucumber (@ourapp\node_modules\@cucumber\cucumber\src\api\run_cucumber.ts:125:19) at Cli.run (@ourapp\node_modules\@cucumber\cucumber\src\cli\index.ts:78:25) at Object.run [as default] (@ourapp\node_modules\@cucumber\cucumber\src\cli\run.ts:38:14) (node:1388) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2) Error: Request failed with status code 404: {"errorCode":4043,"message":"Test Item '43cb9634-3c0a-4b21-b044-66ccb47a01bc' not found. Did you use correct Test Item ID?"} at @ourapp\node_modules\@reportportal\client-javascript\lib\rest.js:37:15 at runMicrotasks () at processTicksAndRejections (internal/process/task_queues.js:95:5) (node:1388) UnhandledPromiseRejectionWarning: Error: Request failed with status code 404: {"errorCode":4043,"message":"Test Item '43cb9634-3c0a-4b21-b044-66ccb47a01bc' not found. Did you use correct Test Item ID?"} at @ourapp\node_modules\@reportportal\client-javascript\lib\rest.js:37:15 at runMicrotasks () at processTicksAndRejections (internal/process/task_queues.js:95:5) (node:1388) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 5) (node:1388) UnhandledPromiseRejectionWarning: Error: Item "bi4rj9d12klbndswus" not found at RPClient.finishTestItem (@ourapp\node_modules\@reportportal\client-javascript\lib\report-portal-client.js:504:47) at CucumberReportPortalFormatter.onTestCaseFinishedEvent (@ourapp\node_modules\@reportportal\agent-js-cucumber\modules\api\current.js:452:23) at EventEmitter. (@ourapp\node_modules\@reportportal\agent-js-cucumber\modules\api\current.js:58:23)
at EventEmitter.emit (events.js:412:35) at TestCaseRunner.runAttempt (@ourapp\node_modules\@cucumber\cucumber\src\runtime\test_case_runner.ts:254:27)
at TestCaseRunner.run (@ourapp\node_modules\@cucumber\cucumber\src\runtime\test_case_runner.ts:185:29) at Runtime.runTestCase (@ourapp\node_modules\@cucumber\cucumber\src\runtime\index.ts:89:20) at Runtime.start (@ourapp\node_modules\@cucumber\cucumber\src\runtime\index.ts:116:7) at runCucumber (@ourapp\node_modules\@cucumber\cucumber\src\api\run_cucumber.ts:125:19) at Cli.run (@ourapp\node_modules\@cucumber\cucumber\src\cli\index.ts:78:25) at Object.run [as default] (@ourapp\node_modules\@cucumber\cucumber\src\cli\run.ts:38:14) (node:1388) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 6) (node:1388) UnhandledPromiseRejectionWarning: Error: Item "bi4rj9d12klbndo8lu" not found at RPClient.finishTestItem (@ourapp\node_modules\@reportportal\client-javascript\lib\report-portal-client.js:504:47)

ddebonotwilio commented 1 year ago

I am facing the same error. I've been trying to narrow the issue down, as it seems to also be related to hooks. But ultimately when I just run 1 scenario (which would anyway trigger the hooks for that 1 scenario), all is fine. If more than 1 scenario are run, the issue mentioned above occurs.

swathisnice commented 1 year ago

@chivekrodis @AmsterGet can we see an update on this.

AmsterGet commented 1 year ago

Hi guys! I got sick and @chivekrodis left the team. I will ask other team members to investigate this issue, hopefully the team can do it this/next week. Sorry for inconvenience.

dwaned commented 1 year ago

Hi @AmsterGet , first of all I hope you get well soon. Secondly, I have uploaded a test repo with how I tried to use this new agent version, in the readme I am mentioning some issues that I have noticed. Please let me know if you notice any mistakes in my code that might be triggering these errors (including the one mentioned in this thread)

Bam6ycha commented 1 year ago

Hi guys! I am trying to figure out what is actually going wrong. I will try to fix that issue as soon as possible. Thank you for being so patient. @dwaned thank you for the test repo, I hope it will help!

Bam6ycha commented 1 year ago

@dwaned @swathisnice @ddebonotwilio hi, everyone! That issue was successfully fixed. We released a new version of our cucumber agent 5.1.1 which contains that fix. In that case, you can update the version of your cucumber agent and try it with that fix.

dwaned commented 1 year ago

Hi @Bam6ycha , happy new year and thank you for the update! I will try it out today and give feedback here. Thank you so much!

Malahovskys commented 1 year ago

v5.1.1 Status of the second scenario is IN_PROGRESS now.

node_modules\@reportportal\client-javascript\lib\report-portal-client.js:451 const err = new Error( ^

Error: Launch with tempId "4x5hx4aolciwrc1c" is already finished, you can not add an item to it at RPClient.startTestItem (\node_modules\@reportportal\client-javascript\lib\report-portal-client.js:451:19) at CucumberReportPortalFormatter.onTestStepStartedEvent (\node_modules\@reportportal\agent-js-cucumber\modules\api\current.js:274:44) at EventEmitter. (\node_modules\@reportportal\agent-js-cucumber\modules\api\current.js:52:23) at EventEmitter.emit (node:events:525:35) at EventEmitter.emit (node:domain:489:12) {

CallSite {}, CallSite {},
CallSite {}, CallSite {},
CallSite {}, CallSite {},
CallSite {}, CallSite {},
CallSite {}, CallSite {}

], [Symbol(mutatedCallSite)]: [ CallSite {}, CallSite {}, CallSite {}, CallSite {}, CallSite {} ] }

dwaned commented 1 year ago

Unfortunately I'm experiencing as well the same issue. What's strange is that:

You can see this from the screenshot showing the failures, then succesful 3 runs, then failures again, on the test repo. image

I also noticed these errors in the Docker logs

2023-01-05 12:33:47 reportportal-api-1               | 2023-01-05 10:33:47.937 ERROR 1 --- [nio-8585-exec-1] .AuthUtils$SerialUidReplacingInputStream : Potentially Fatal Deserialization Operation.
2023-01-05 12:33:47 reportportal-api-1               | 
2023-01-05 12:33:47 reportportal-api-1               | java.io.InvalidClassException: Overriding serialized class version mismatch: local serialVersionUID = 550 stream serialVersionUID = 520
2023-01-05 12:33:47 reportportal-api-1               |  at com.epam.ta.reportportal.auth.util.AuthUtils$SerialUidReplacingInputStream.readClassDescriptor(AuthUtils.java:105)
...
Bam6ycha commented 1 year ago

@Malahovskys @dwaned This is terrible news. I'm sorry it happened. I'll take care of this problem right now. I hope you understand and thank you for your patience.

dwaned commented 1 year ago

Also, maybe its helpful, my error in the console log is like this

> cd .dist && mkdir -p report && ../node_modules/@cucumber/cucumber/bin/cucumber-js -v && ../node_modules/@cucumber/cucumber/bin/cucumber-js -require=/**/*.js --format summary ./features -f ./step_definitions/support/reportPortal/reportPortalFormatter.js

8.9.1
/Users/ddebono/code/Projects/test-rp-new/node_modules/@reportportal/client-javascript/lib/report-portal-client.js:473
          new Error(`Item with tempId "${parentTempId}" not found`),
          ^

Error: Item with tempId "sktwimlglcizpwhb" not found
    at RPClient.startTestItem (/Users/ddebono/code/Projects/test-rp-new/node_modules/@reportportal/client-javascript/lib/report-portal-client.js:473:11)
    at CucumberReportPortalFormatter.onTestStepStartedEvent (/Users/ddebono/code/Projects/test-rp-new/node_modules/@reportportal/agent-js-cucumber/modules/api/current.js:274:44)
    at EventEmitter.<anonymous> (/Users/ddebono/code/Projects/test-rp-new/node_modules/@reportportal/agent-js-cucumber/modules/api/current.js:52:23)
    at EventEmitter.emit (node:events:549:35)
    at Coordinator.parseWorkerMessage (/Users/ddebono/code/Projects/test-rp-new/node_modules/@cucumber/cucumber/lib/runtime/parallel/coordinator.js:64:35)
    at ChildProcess.<anonymous> (/Users/ddebono/code/Projects/test-rp-new/node_modules/@cucumber/cucumber/lib/runtime/parallel/coordinator.js:101:18)
    at ChildProcess.emit (node:events:537:28)
    at emit (node:internal/child_process:937:14)
    at process.processTicksAndRejections (node:internal/process/task_queues:83:21)

Node.js v18.5.0

and sometimes, the node process gets stuck like this:

➜  node:events:515
      throw er; // Unhandled 'error' event
      ^

Error [ERR_IPC_CHANNEL_CLOSED]: Channel closed
    at new NodeError (node:internal/errors:388:5)
    at target.send (node:internal/child_process:740:16)
    at Worker.sendMessage (/node_modules/@cucumber/cucumber/lib/runtime/parallel/run_worker.js:18:43)
    at EventEmitter.<anonymous> (/node_modules/@cucumber/cucumber/lib/runtime/parallel/worker.js:31:18)
    at EventEmitter.emit (node:events:537:28)
    at TestCaseRunner.aroundTestStep (/node_modules/@cucumber/cucumber/lib/runtime/test_case_runner.js:130:31)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async TestCaseRunner.runAttempt (/node_modules/@cucumber/cucumber/lib/runtime/test_case_runner.js:157:13)
    at async TestCaseRunner.run (/node_modules/@cucumber/cucumber/lib/runtime/test_case_runner.js:135:35)
    at async Worker.runTestCase (/node_modules/@cucumber/cucumber/lib/runtime/parallel/worker.js:83:9)
Emitted 'error' event on process instance at:
    at node:internal/child_process:744:35
    at process.processTicksAndRejections (node:internal/process/task_queues:77:11) {
  code: 'ERR_IPC_CHANNEL_CLOSED'
}

Node.js v18.5.0

then having to CTRL+C to break the process.

dwaned commented 1 year ago

Hi @Bam6ycha any news on this?

Bam6ycha commented 1 year ago

Hi, @dwaned. I have some updates in that case. It looks like you are trying to start reporting in parallel mode. Unfortunately, our agent doesn't support that feature now. At the moment, the agent supports sequential reporting, try to add --format-options '{"snippetInterface": "synchronous"} in cucumber config, and remove --parallel option. I hope it will help. In my case, the mentioned error occurs only, when I turn on --parallel option.

In the nearest future, we are planning to update our agent for support parallel reporting as well.

dwaned commented 1 year ago

Thank you @Bam6ycha I will try to do that. At the moment even though there is --parallel, it was running with 1 Cucumber Worker so essentially it shouldn't be running anything in parallel. But I will try it out and give feedback here.

Bam6ycha commented 1 year ago

@dwaned, if you face the same error again, can you provide a reproducible example? Because I can't even reproduce it, the only case when such error occurs it is when --parallel is turned on.(in my case)

marianolopezcortes commented 1 year ago

Hi @Bam6ycha. I'm the same Interrupted behavior with 2 scenario feature, with parallel disabled and version 5.1.1

Feature: I can login

Background: Given I am on the login page

Scenario: I verify that form login is Ok Then the login form should be displayed correctly

Scenario: I can login as admin When I log as admin Then I am on the Home Page

Bellow my trace:

VError: an AfterAll hook errored, process exiting: src\support\common-hooks.ts:53: Item with tempId "zpwcqsldsrikph" not found at Runtime.runTestRunHooks (C:\Users---\dev\qa-e2e-frontv2\node_modules\@cucumber\cucumber\src\runtime\run_test_run_hooks.ts:32:19) at processTicksAndRejections (node:internal/process/task_queues:95:5) at Runtime.start (C:\Users---\dev\qa-e2e-frontv2\node_modules\@cucumber\cucumber\src\runtime\index.ts:118:5) at runCucumber (C:\Users---\dev\qa-e2e-frontv2\node_modules\@cucumber\cucumber\src\api\run_cucumber.ts:125:19)

In my AfterAll hook I close my browser:

AfterAll(async function () { await browser.close(); });

Any ideas? Thanks!!

MatanNoachNice commented 1 year ago

Hi,

I am facing similar issue with feature files with 2 scenarios (or more), but with different behavior:

On the second scenario, I am getting this error:

@MyAppDirectory\node_modules\@reportportal\client-javascript\lib\report-portal-client.js:240
    Promise.all(launchObj.children.map((itemId) => this.map[itemId].promiseFinish)).then(
                                                                    ^
TypeError: Cannot read properties of undefined (reading 'promiseFinish')]
    at @MyAppDirectory\node_modules\@reportportal\client-javascript\lib\report-portal-client.js:240:69
    at Array.map (<anonymous>)
    at RPClient.finishLaunch (@MyAppDirectory\node_modules\@reportportal\client-javascript\lib\report-portal-client.js:240:36)
    at @MyAppDirectory\node_modules\@reportportal\agent-js-cucumber\modules\api\current.js:496:25
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

And on the Report Portal I see that the test still run (even though the process has finished), and that the 2nd scenario contains only some of the steps but they all passed.

BTW, I am also using --format-options '{"snippetInterface": "synchronous"}' option

Bam6ycha commented 1 year ago

@marianolopezcortes, @MatanNoachNice Hi there. Guys, it would be nice to provide repo with reproducible examples for a more effective investigation. It will help me a lot to figure it out. Because in most cases, I spend a lot of time reproducing the issue. I hope you will understand.

marianolopezcortes commented 1 year ago

Hello @Bam6ycha. Here is my repository:

https://github.com/marianolopezcortes/e2e-cucumber-playwright.git

Cucumber + playwright simple project

You have to set RP_URL and RP_TOKEN with your server URL and UUID in config/default.json.

The error is as shown in my previous message:

VError: an AfterAll hook errored, process exiting: src\support\common-hooks.ts:53: Item with tempId "zpwcqsldsrikph" not found at Runtime.runTestRunHooks (C:\Users---\dev\qa-e2e-frontv2\node_modules@cucumber\cucumber\src\runtime\run_test_run_hooks.ts:32:19) at processTicksAndRejections (node:internal/process/task_queues:95:5) at Runtime.start (C:\Users---\dev\qa-e2e-frontv2\node_modules@cucumber\cucumber\src\runtime\index.ts:118:5) at runCucumber (C:\Users---\dev\qa-e2e-frontv2\node_modules@cucumber\cucumber\src\api\run_cucumber.ts:125:19)

2nd scenario is never executed. No problem for a 1-scenario feature. No matter if parallel is activated or not.

If you need some other information, let me know :smiley:.

Thx!

marianolopezcortes commented 1 year ago

Hello again @Bam6ycha. I've just tested the 5.2.0 version and all is Ok. You are great! Thank you all!

Bam6ycha commented 1 year ago

@marianolopezcortes, you are welcome!

dwaned commented 1 year ago

I can confirm this is also now working correctly for me too. Well done ppl!

MatanNoachNice commented 1 year ago

Hi @Bam6ycha Sorry for not answering to your previous message. But I also wanted to inform that the issue seems to be fixed in version 5.2.0 and cucumber 8.4.0 Thank you everyone for your help!

Bam6ycha commented 1 year ago

It looks like we've done with this issue. And now we can close it. Thank you all for your time and help! Good luck!