Allure report treats passed Mocha beforeAll and afterAll hooks as passed tests

[MartijnThiele]

I am building a ta-framework using

-- wdio-mocha-framework@0.6.2 -- wdio-allure-reporter@0.6.3 `-- webdriverio@4.13.1

My framework has around 20 spec files, most of which make use of mocha's beforeAll and afterAll hooks to do setup and cleanup.

My Allure report (generated by jenkins) sees these hooks as actual tests, reporting them as such. This causes allure to report almost 40 extra tests, which messes up my report

I also found a variation to this behaviour: if a spec file contains one or more skipped tests, the beforeAll method will show up as skipped also. This could have something to do with the freshly fixed bug that causes only 1 skipped test to be shown, but I am not sure.

For reference, the images below are allure report and wdio spec report for the same test run.

Could the wdio allure reporter be adjusted so that beforeAll and AfterAll hooks are not shown anymore when passing? (a failing hook is a different story, the default mocha way of handling these could use some improvement too)

[BorisOsipov]

@MartijnThiele

create sample project on github to help me reproduce these issues

[MartijnThiele]

@BorisOsipov

created a sample project here:

https://github.com/MartijnThiele/wdioAllureSampleProjectIssue-128

Thanks for taking the time to look into this.

Please let me know if there are any questions

[BorisOsipov]

@MartijnThiele thank you. I will have look into it

[BorisOsipov]

yes, it is a bug.

[MartijnThiele]

I suspected as much. Anything you can fix?

[BorisOsipov]

I think we can disable\enable reporting hooks at all with configuration option.

Anything you can fix?

I am not sure. I don't have plans to support this version much. Recently I port this to v5 and will continue adding fixes\improvements only for v5

[MartijnThiele]

Pity, can you point me towards this configuration option? I am not sure what you mean by this.

[BorisOsipov]

I mean if you want to make PR to fix this bug, I suppose the best solution is to create a new option to disable reporting hooks at all.

[MartijnThiele]

OK, thanks for the info. That would sound like a good solution. I'm afraid I am not experienced enough to solve something like this myself, but I will have a go and see what I can do.

[sanyco92]

Hello guys! Is this issue fixed? I'm experiencing the same issue. Hooks "before all" and other are marked as test cases and including to the total number of tests.

[BorisOsipov]

@sanyco92

Is this issue fixed?

nope

[brownad]

Can this be ported? https://github.com/allure-framework/allure-js/commit/cbbb4491d4d4c8a81b58aed1d28210d4d0cfb75d

[christian-bromann]

Can this be ported?

Feel free to provide a PR

[BorisOsipov]

It is fixed in v5

[brownad]

unfortunately still stuck on v4 for a while longer thanks for the update @BorisOsipov

[christian-bromann]

still stuck on v4 for a while longer

Why is that? It is really not much work to update.

[brownad]

v4 is wrapped up in our own tooling suite in a way that requires more work than anticipated to move but, that was based on beta v5 last time we looked. Going to start the move / assessment again. thanks @christian-bromann

[just-boris]

Can this be ported? allure-framework/allure-js@cbbb449

@brownad the commit fixed the handling for beforeEach/afterEach hooks. That was a regression in 2.0.0 version comparing to 1.x. The before and after hooks are not addressed yet.

I created an issue to track this work: https://github.com/allure-framework/allure-js/issues/48

P.S. Please note that 2.0 version is not yet stable and there may be breaking changes. I advice not to use the dependency in your projects until the stable release is shipped.

[qwez]

It is fixed in v5

@BorisOsipov I use v5 and still have this issue

"devDependencies": {
    "@wdio/allure-reporter": "^5.12.1",
    "webdriverio": "^5.12.5"
  }

My report:

[BorisOsipov]

fix @qwez relesed in v5.13.0