allure-framework / allure-js

Allure integrations for JavaScript test frameworks
https://allurereport.org/
Apache License 2.0
225 stars 121 forks source link

Tracing Options (--steps/--debug/--verbose) No Longer Provide Step-by-Step Output Starting from "allure-codeceptjs v3.0.0-beta.10" #1167

Open zyab1ik opened 5 days ago

zyab1ik commented 5 days ago

Describe the bug In allure-codeceptjs 3.0.0-beta.10, when running tests with the --steps or --debug options, the expected step-by-step output is no longer shown

To Reproduce Steps to reproduce the behavior:

  1. Run any test with the --steps or --debug option
  2. Observe that the expected step-by-step output is not displayed.

Expected behavior Tests should display step-by-step reports with trace options

Screenshots "allure-codeceptjs": "3.0.0-beta.10":

image

"allure-codeceptjs": "3.0.0-beta.9"

image image

Additional context This issue started with allure-codeceptjs version 3.0.0-beta.10 and persists in the latest version. The expected step-by-step output for the --steps and --debug options does not work

baev commented 3 days ago

@zyab1ik Since 3.0.0-beta.10, allure-codeceptjs integration was reworked. Now, it uses a Mocha reporter underneath (taking allure-mocha as a baseline) and enriches the report using some Codeceptjs events.

I honestly don't know the --steps or --debug options you're talking about. Could you please elaborate and explain them in more detail?

zyab1ik commented 3 days ago

@baev In CodeceptJS, there are standard options for debugging tests, such as --steps, --debug, and --verbose. More details can be found in their documentation: #debug option commands

However, when using the Allure adapter together with CodeceptJS above the version allure-codeceptjs v3.0.0-beta.9, these options stop working. If you disable the Allure adapter through the config or downgrade the version to allure-codeceptjs v2.15.1, the detailed test output in the terminal works fine. Version of codeceptJS - doesn't matter. I've chcked on vesrions started from 3.6.2 and up.

Could you suggest a solution to this? Is there any way to fix it?

P/s

image image
baev commented 2 days ago

@zyab1ik thanks

According to https://github.com/codeceptjs/CodeceptJS/issues/698#issuecomment-327282346, the console output will not work if a custom reporter is used.

Since Allure currently overrides the specified reporter configuration, the options mentioned above will not work.

The CodeceptJS team suggests using the mocha-multi reporter, which will not work with Allure CodeceptJS since it relies on CodeceptJS events as well ( Allure CodeceptJS is a Mocha reporter and CodeceptJS plugin at the same time).

We plan to provide mocha-multi functionality in Allure Mocha (so you can add Allure Mocha reporter & proxy events to some 3rd party reporters). Using this feature, it may be possible to enable codeceptjs/lib/cli reporter by default, which should fix the issue.