allure-framework / allure-js

Allure integrations for JavaScript test frameworks
https://allurereport.org/
Apache License 2.0
225 stars 121 forks source link

allure-mocha: add possibility to attach a screenshot scoped to a failing test only #236

Closed dmytro-grablov closed 3 months ago

dmytro-grablov commented 3 years ago

[//]: # ( . Note: for support questions, please use Stackoverflow or Gitter. . This repository's issues are reserved for feature requests and bug reports. . . In case of any problems with Allure Jenkins plugin please use the following repository . to create an issue: https://github.com/jenkinsci/allure-plugin/issues . . Make sure you have a clear name for your issue. The name should start with a capital . letter and no dot is required in the end of the sentence. An example of good issue names: . . - The report is broken in IE11 . - Add an ability to disable default plugins . - Support emoji in test descriptions )

I'm submitting a ...

What is the current behavior?

Given current mocha root hook

import allureRuntime from 'allure-mocha/runtime.js';
import { ContentType } from 'allure-js-commons';

async function afterEach(this: Context): Promise<void> {
    const allure = allureRuntime.allure;
    if (this.currentTest?.err) {
        try {
                await allure.createAttachment('Screenshot', await page.screenshot(), ContentType.PNG);
            } catch (err) {
                const message = `could not create a screenshot. Reason: ${err.message}`;
                await allure.createAttachment('Screenshot', message, ContentType.TEXT);
            }
    }
}

And a suite of 3 tests When tests # 1 and # 2 fail with different reasons, # 3 passes Then each of 3 tests in the report contains 2 attachments in the 'Tear Down' block.

What is the expected behavior?

Then tests # 1 and # 2 contain only their respective attachments, test # 3 contains no attachments at all.

What is the motivation / use case for changing the behavior?

Clean reports. Tests are independent from each other, so when one fails I don't want to see its attachment in another test report. Currently the only way I found to attach something exactly to a test is doing this within the test body, but then the code is not reusable. When allure is called from afterEach, block the currentTest is already not accessible in AllureReporter although it is accessible from mocha context.

Please tell us about your environment:

Test framework mocha@8.3.0
Allure adaptor allure-mocha@2.0.0-beta.9
Generate report using allure-console@2.13.3

Other information

[//]: # ( . e.g. detailed explanation, stacktraces, related issues, suggestions . how to fix, links for us to have more context, eg. Stackoverflow, Gitter etc )

iqbalsingh-qa commented 3 years ago

Hi @dmitriy-grablyov, I also facing the same problem. I want to add a screenshot for all the test cases in the allure report. but for the very first test case screenshot is not added instead screenshot of the first one added to the next one and the same is happening with all others and the last test case contains 2 screenshots. The same thing happens if I want to add screenshots only for failures. A screenshot of failed one is added to the next test case in the report. I'm adding code for screenshot in afterEach hook I think this is happening only due to this.

afterEach(function () {
            driver.takeScreenshot().then(function (image, err) {
                allure.createAttachment('Screenshot', Buffer.from(image, "base64"), 'image/png');
            });
        })

So, do you have an idea about how I can correct it? A screenshot of the correct test case should be added to the correct one.

ostrovanka commented 3 years ago

Hi, are there any plans to update the described behaviour? Current implementation is very inconvenient, as mostly screenshots for UI tests are usually taken and attached in the afterEach hooks. Currently all screenshots taken after all failed tests will be attached to all teat down blocks for all tests. This is definitely not what most people will expect here. :)

zhq745w commented 2 years ago

https://github.com/allure-framework/allure-mocha had been deprecated. And now this issue is very actually

zhq745w commented 6 months ago

up?

baev commented 6 months ago

We're currently working on the major commons refactoring, that will address this issue among the others. See https://github.com/allure-framework/allure-js/pull/909