allure-framework / allure-js

Allure integrations for JavaScript test frameworks
https://allurereport.org/
Apache License 2.0
226 stars 120 forks source link

"MochaAllure: "step" can't be used in parallel mode!" log while starting test in single thread mode #753

Closed IvanRogovskiy closed 3 months ago

IvanRogovskiy commented 1 year ago

Describe the bug When I start test in single thread mode, i see the log with the following message:

MochaAllure: "step" can't be used in parallel mode! To use Allure Runtime API, please, switch back to single thread mode.

And after this test fails.

Mocha config:

module.exports = {
  require: ['ts-node/register', 'dotenv/config', './scripts/mochaHooks.ts', 'mocha-allure-reporter'],
  reporter: 'mocha-multi-reporters',
  reporterOptions: {
    configFile: './reporterConfig.json',
  },
  timeout: 60000,
  parallel: false,
  exit: true,
};

As you can see, mocha configured in a single thread mode, while MochaAllure reports that parallel mode enabled.

To Reproduce Steps to reproduce the behavior:

  1. Start test in a single thread mode
  2. Watch logs

Desktop (please complete the following information):

baev commented 1 year ago

What version of allure-mocha are you using?

IvanRogovskiy commented 1 year ago

version 2.5.0

baev commented 1 year ago

Could you please share a sample project in the form of a GitHub repo to reproduce the problem?

IvanRogovskiy commented 1 year ago

@baev I added you as collaborator to my sample repo, where error is reproducible, take a look please

epszaw commented 1 year ago

Have discovered, a moment ago, that the reporter works correctly without mocha-multi-reporters. It's pretty strange because we use mocha-multi-reporters testing the reporter itself and it works.

As a temp solution, I can propose to use only allure-mocha reporter.

Keep searching for the true solution.

epszaw commented 1 year ago

I didn't find any problem with allure-mocha, but mocha-multi-reporters doesn't react to the config changes and doesn't see allure-mocha reporter, it always has spec,xunit reporters. You can check it locally by yourself.

  1. Open node_modules/mocha-multi-reporters/lib/MultiReporters.js
  2. Add console.log or debugger statement on line 89
  3. Check reporterEnabled field in the options

There is a screenshot with an attached debugger:

image

The mocha config looks in this way:

image
epszaw commented 1 year ago

You can see a proposed solution here. The issue relates to mocha-multi-reporters and the issue has been created many time ago.

IvanRogovskiy commented 1 year ago

I turned off multi-reporters and still the same issue. By the way, it happens only when i use step() function. If i don't use it everything works fine, but i need to define my own steps. I also checked the pid when allureGetter() is called and it's the same as pid of tests run. (to exclude the fact of multithreading).

image

my .mocharc

image

epszaw commented 1 year ago

Tried to use step in the code with configuration you send above and it works:

describe('Dummy', async () => {
    it('Dummy test' ,async () => {
        testLogger.info('Step 1. Generate random string')
        const dummyId = generateRandomNUMString(1, 10);
        testLogger.info(dummyId)

        testLogger.info('Step 2. Send get with random string')
        expect(1).eq(1)
        const resp = await axiosHelper.sendGetToUrl(`https://dummy.restapiexample.com/api/v1/employee/${dummyId}`)

+   await allure.step("example step", async () => {
+           return true
+   })

        testLogger.info('Step 3. Check id the in response')
        expect(resp.data.data.id).eq(Number(dummyId))
    })
})

The test result file is:

{
  "uuid": "cd650e77-5749-4df1-9483-bd1687728bd4",
  "historyId": "6baf33fa583578a84b20a7b948ddba0b",
  "status": "broken",
  "statusDetails": {
    "message": "Cannot read properties of undefined (reading 'data')",
    "trace": "TypeError: Cannot read properties of undefined (reading 'data')\n    at Context.<anonymous> (tests/DummySuite/dummy.spec.ts:22:21)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)"
  },
  "stage": "finished",
  "steps": [
    {
      "statusDetails": {},
      "stage": "running",
+      "steps": [
+        {
+          "status": "passed",
+          "statusDetails": {},
+          "stage": "finished",
+          "steps": [],
+          "attachments": [],
+          "parameters": [],
+          "start": 1692694404826,
+          "name": "example step"
+        }
+      ],
      "attachments": [],
      "parameters": [],
      "start": 1692694404817,
      "name": "sendGetToUrl"
    }
  ],
  "attachments": [],
  "parameters": [],
  "labels": [
    {
      "name": "package",
      "value": "tests.DummySuite"
    },
    {
      "name": "parentSuite",
      "value": "Dummy"
    }
  ],
  "links": [],
  "start": 1692694404816,
  "name": "Dummy test",
  "fullName": "Dummy test",
  "stop": 1692694404828
}
IvanRogovskiy commented 1 year ago

By the way, it works only when start tests by command: mocha "tests/hourly/*/.spec.ts" for example.

If i start test with IDE (WebStorm) "Run test" button (on the screenshot below), error remains.

image

Winify commented 10 months ago

is there some update on this? I experience the same issue

I do not use mocha-multi-reporters and the problem persists if I only use allure-mocha as a reporter as well.

epszaw commented 10 months ago

is there some update on this? I experience the same issue

I do not use mocha-multi-reporters and the problem persists if I only use allure-mocha as a reporter as well.

Could you send your mocha config there?

Winify commented 10 months ago

@epszaw

.mocharc.cjs

module.exports = {
    require: [
        'ts-node/register',
        'chai/register-expect',
    ],
    loader: [
        'ts-node/esm',
        'ts-paths-esm-loader/transpile-only'
    ],
    extension: 'ts',
    ui: 'bdd',
    timeout: 60000,
    recursive: true,
    spec: [
        'test/specs/**/*.spec.ts'
    ],
    parallel: true,
    reporter: 'allure-mocha'
}

I use ts-node/typescript to write my tests and my package.json is type: "module", so the mocha config file is a .cjs as the documentation states.

Log:

> mocha --parallel

(node:9160) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
(node:2952) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
MochaAllure: "step" can't be used in parallel mode! To use Allure Runtime API, please, switch back to single thread mode.
MochaAllure: "step" can't be used in parallel mode! To use Allure Runtime API, please, switch back to single thread mode.
MochaAllure: "step" can't be used in parallel mode! To use Allure Runtime API, please, switch back to single thread mode.

Process finished with exit code 2
richardwallispr commented 6 months ago

Also having this issue, the log file is being peppered with errors anytime an allure.function is being called, despite the fact that parallel = false in mocharc config.

The allure calls like allure.suite or allure.attachment work, they're being added to the report this however:


it("steps", () => {
  allure.step("Step 1", () => {
    allure.step("Sub-step 1", () => {
      // ...
    });
    allure.step("Sub-step 2", () => {
      // ...
    });
  });
  allure.step("Step 2", () => {
    // ...
  });
});

won't work, anything code inside the step just won't run, it's completely bypassed when debugging

delatrie commented 3 months ago

Hi, everyone! We've changed the mechanism allure-mocha uses to handle parallel mode and how the runtime API is exposed in version 3.0 (currently, 3.0.0-beta.3 is accessible). I was unable to reproduce the issue, so chances are high that it's resolved.

If that's not the case and you're still facing problems accessing the API, please open a new issue and provide an example to reproduce.