Shelex / cypress-allure-plugin

cypress plugin to use allure reporter api in tests
https://shelex.github.io/cypress-allure-plugin-example/
Apache License 2.0
159 stars 44 forks source link

Generate tests in forEach loop mark first loop with tests as skipped #188

Closed PeaceDate closed 1 year ago

PeaceDate commented 1 year ago

Reopened #183

Updated to latest version of Cypress allure plugin. Problem still reproduce

Steps to reproduce

  1. Clone this repo
  2. Run yarn install
  3. Run npm run cypress:test
  4. Run npm run report:generate
  5. Run npm run report:open

Result -> First test ignored and marked as skipped

Screenshot 2023-02-02 at 22 06 09

@shelex/cypress-allure-plugin: "2.35.0" cypress: "10"

Shelex commented 1 year ago

@PeaceDate Thank you for an example repo, that helped a lot.

Now I see that problem is not a forEach loop actually, but the changing domain, so this issue is identical to https://github.com/Shelex/cypress-allure-plugin/issues/179 and https://github.com/Shelex/cypress-allure-plugin/issues/125.

It is caused by a way Cypress work - baseUrl is absent, same file involving different domains (technically, localhost:3001 and localhost:3002 are different), thus Cypress reloads the page completely and erases previous allure object.

I don't see any quick solution for this, as there is no clear way how to identify that Cypress decided to clear the state, so we could keep reporter object in some safe place, and then put into the new window state.

PeaceDate commented 1 year ago

@Shelex That make sence. I haven't figured out. Thank you a lot :)

Shelex commented 1 year ago

UPD: What happens in case of changing domains:

Without domain changing cypress fires "test pending" event only in case it is going to be skipped. So in v2.35.1 I upgraded handler for "end test" event, and in case test was not skipped - set proper status. However, as state was cleared, test object is empty, meaning there will be no steps attached - that is still to be investigated how to resolve.

PeaceDate commented 1 year ago

@Shelex HI, I checked and it works as you said. Thank you a lot, I'll continue monitoring this question for new changes and no steps attached issue.

Have a good day and Take care of yourself : )

PeaceDate commented 1 year ago

@Shelex Isn't working with Mocha.Context.skip() when it add like Cypress custom command Get an error TypeError: Cannot read properties of undefined (reading 'state') Because this error occurred during a after all hook we are skipping all of the remaining tests.

Cypress.Commands.add('skipWhen', function (condition) {
    if (condition) {
        this.skip()
    }
})

Added such an example in the repo called skipTestwithSkipWhen.cy.ts

Shelex commented 1 year ago

@PeaceDate case with optional skip is fixed in v2.35.2

PeaceDate commented 1 year ago

@Shelex Thank you, I've checked already. Everything is working fine 🔥