search

Shelex / cypress-allure-plugin

cypress plugin to use allure reporter api in tests
https://shelex.github.io/cypress-allure-plugin-example/
Apache License 2.0
159 stars 44 forks source link

Allure-results can't be created if "context.skip" or "describe.skip" is presented in suite body #65

Closed Seitar18 closed 3 years ago

Seitar18 commented 3 years ago

Allure-results can't be created for spec file if "context.skip" or "return this.skip();" at "before" hook are presented in suite body so it doesn't even appears as tests at allure report results when all specs passed. Mocha/mochawesome handle this correctly but this plugin unfortunately not.

Steps to reproduce:

  1. Use modified attached* cypress/examples/actions.spec file at cloned project(I've just remove all(except the first) actions from your original file for more speed and results visibility; reproduced the same at production with big number of tests)
  2. run "yarn cy:run" at project directory
  3. Observe the result

Expected result Plugin should correctly handle test statements and create corresponding test result:

Actual result

  1. https://pastebin.com/4EK0uYcF - context.skip example actions.spec file with partially skipped tests https://pastebin.com/hq7Xa59r- console output Allure results wasn't created
  2. https://pastebin.com/cYfmYbNF - another context.skip example actions.spec file with all spec.file tests skipped hhttps://pastebin.com/1tC2inqjr- console output Allure results wasn't created

Environment :

Shelex commented 3 years ago

@Seitar18 Thank you for opening issue. Actually it is known one. I have no idea why Cypress after receiving mocha "pending" event (that happens when next test\suite should be skipped) and if there is no other tests\hooks left - just stops cypress runner and as a result no commands could be used, so results are not saved.

It will require proper investigation if this could be fixed or workaround found. I assume empty after hook may work in case you don't have any hooks yet.

Seitar18 commented 3 years ago

@Shelex thank you for your answer. Unfortunately we have many before hooks at those tests. So, may be global after hook should help?

Shelex commented 3 years ago

@Shelex thank you for your answer. Unfortunately we have many before hooks at those tests. So, may be global after hook should help?

When I was debugging such issue it helped me. When any other executable is scheduled after skipped test - allure results writer is fine.

Seitar18 commented 3 years ago

@Shelex thank you for your answer. Unfortunately we have many before hooks at those tests. So, may be global after hook should help?

When I was debugging such issue it helped me. When any other executable is scheduled after skipped test - allure results writer is fine.

Thanks a lot[2]! This call at support/index.js is solved this problem. after(function () { });

Shelex commented 3 years ago

@Seitar18 glad it helped! However it should be addressed properly, so let's leave issue opened.

Shelex commented 3 years ago

Addressed in 2.6.0.