allure-framework / allure-js

Allure integrations for JavaScript test frameworks
https://allurereport.org/
Apache License 2.0
230 stars 124 forks source link

allure-cucumberjs: report skipped scenarios in case of before hook failure #907

Closed baev closed 5 months ago

baev commented 8 months ago

Discussed in https://github.com/orgs/allure-framework/discussions/2410

Originally posted by **devrajshetake** March 11, 2024 I am using Cucumber and Playwright with typescript to run my BDD tests. The allure report I am generating contains all the executed scenarios, but it does not contain the scenarios which are not executed. A `BeforeAll` hook runs before execution of all the scenarios within a tag, the problem arises when the `BeforeAll` hook fails for some reason, all the scenarios within this tag wont be executed, so they wont be mention in the report. Due to this, the total number of scenarios within my test suit are being displayed less in the report. I want the unexecuted scenarios to be mentioned in the allure report. Is there a way to do so? Please help.
baev commented 5 months ago

@devrajshetake Currently, exceptions in BeforeAll aren't handled properly by cucumberjs

Error: Unexpected error on worker.receiveMessage
    at exit (node_modules/@cucumber/cucumber/lib/runtime/parallel/run_worker.js:11:27)
    at node_modules/@cucumber/cucumber/lib/runtime/parallel/run_worker.js:24:31 {
  [cause]: Error: a BeforeAll hook errored on worker 2, process exiting: features/support/beforeAllFailure.js:3
      at Worker.runTestRunHooks (@cucumber-cucumber-npm-10.8.0-3268c4cd89-537bd8b891.zip/node_modules/@cucumber/cucumber/lib/runtime/run_test_run_hooks.js:24:23)
      at async Worker.initialize (@cucumber-cucumber-npm-10.8.0-3268c4cd89-537bd8b891.zip/node_modules/@cucumber/cucumber/lib/runtime/parallel/worker.js:53:9)
      at async Worker.receiveMessage (@cucumber-cucumber-npm-10.8.0-3268c4cd89-537bd8b891.zip/node_modules/@cucumber/cucumber/lib/runtime/parallel/worker.js:62:13) {
    [cause]: Error: before all error

So, Allure can't do much about it. Feel free to open a bug report/feature request for cucumber: https://github.com/cucumber/cucumber-js

baev commented 5 months ago

I would suggest you avoid throwing exceptions in BeforeAll hooks, instead, you can use world parameters to pass the error to before hook and throw an exception in there:

BeforeAll(function () {
  try {
    doSomethingWithPossibleException();
  } catch (e) {
     this.parameters.error = e;
  }
});

Before(function () {
  if (this.parameters.error) {
    throw this.parameters.error;
  }
});