Shelex / cypress-allure-plugin

cypress plugin to use allure reporter api in tests
https://shelex.github.io/cypress-allure-plugin-example/
Apache License 2.0
159 stars 44 forks source link

Allure results are not generated for the last feature file that is executed #180

Closed swatinigam closed 1 year ago

swatinigam commented 1 year ago

Describe the bug

Hi, when I run multiple feature files, e.g.4-6 feature files, I noticed that the allure results file is not generated for the last feature file that is executed. When I use DEBUG=allure-plugin* in my npx command, I can confirm that the allure results files were never generated.

To Reproduce Steps to reproduce the behavior:

  1. Run multiple feature files, I ran 6 features i.e. a total of 44 scenarios using the command: DEBUG=allure-plugin npx cypress run --spec 'cypress/e2e/s.feature' --headless --env allure=true allureReuseAfterSpec=true

  2. Then see how after every test, its allure results files are generated. Then wait for the last test to complete running

  3. Note that after the last feature is executed, there is NO logging at all like "allure-plugin:writer starting writing allure results to " which would have indicated that the allure results files were generated for the last test. Instead, all you see is the spec output for all tests runs. Screenshot attached shows the feature's spec output followed by the entire execution's spec output but no allure-writer logs in between the two.

  4. Go to the allure-results folder to confirm that allure results files were not generated for the last feature. I would say that this happens 90% of the time. And I didn't change anything when it did work as expected.

  5. Note that in the scrrenshot attached, the generated allure report only has 38 tests instead of 44 that were executed. This is because the last test was executed but its allure files were not created.

Expected behavior I expect to see allure-result files generated for each feature, every time the tests are run.

Environment (please complete the following information):

Developer Tools Console Output

Additional context cypress.config.js file is below:

`const { defineConfig } = require('cypress'); const allureWriter = require('@shelex/cypress-allure-plugin/writer'); const cucumber = require('cypress-cucumber-preprocessor').default;

module.exports = defineConfig({ video: false, defaultCommandTimeout: 25000, numTestsKeptInMemory: 0, pageLoadTimeout: 25000, responseTimeout: 25000, screenshotsFolder: 'cypress/screenshots', reporter: 'cypress-multi-reporters', chromeWebSecurity: false, 'cypress-cucumber-preprocessor': { nonGlobalStepDefinitions: true, stepDefinitions: 'cypress/e2e', }, e2e: { setupNodeEvents(on, config) { on('file:preprocessor', cucumber()); allureWriter(on, config); return config; }, baseUrl: 'someUrl', specPattern: ['cypress/e2e//.feature'], }, env: { TAGS: 'not @wip', allure: true, allureReuseAfterSpec: true, allureResultsPath: './allure-results', }, });

`

here are the relevant dependencies: "@shelex/cypress-allure-plugin": "^2.34.0", "allure-commandline": "^2.20.1", "cypress": "^10.11.0", "cypress-cucumber-preprocessor": "^4.2.0",

Has anyone run into this issue or is there a workaround for it? If not, would it be possible for you to investigate?

I am going to see if this also happens in cypress with mocha. Would really love to be able to adopt this in our projects but this issue would be a dealbreaker.

Thank you for your time. Swati
Screen Shot 2023-01-07 at 4 16 37 PM

Screen Shot 2023-01-07 at 4 17 11 PM

Shelex commented 1 year ago

@swatinigam as far as I know cypress-cucumber-preprocessor is not supported anymore as it was deprecated and ownership transferred to @badeball/cypress-cucumber-preprocessor. In case there would still be such issue after migration - please reopen this ticket with example repo where I can reproduce it.

swatinigam commented 1 year ago

@swatinigam as far as I know cypress-cucumber-preprocessor is not supported anymore as it was deprecated and ownership transferred to @badeball/cypress-cucumber-preprocessor. In case there would still be such issue after migration - please reopen this ticket with example repo where I can reproduce it.

Hey @Shelex just wanted to close the loop- I did rerun the tests with @badeball/cypress-cucumber-preprocessor several times and I was not able to reproduce the issue. Thank you for your suggestion and for your help.