search

allure-framework / allure-js-commons

Deprecated, use https://github.com/allure-framework/allure-js instead
Other
27 stars 40 forks source link

Implement custom categories from Allure 2 Beta 6 release #31

Closed doppelganger23 closed 1 year ago

doppelganger23 commented 7 years ago

It would be very nice if you implement support of broken test statuses and a way to determine them via categories.json, as described in this article: https://qameta.io/blog/2017/03/allure2-beta6/

categories 
[
  {
    "name": "Ignored tests",
    "messageRegex": ".*ignored.*",
    "matchedStatuses": [ "skipped" ]
  },
  {
    "name": "Infrastructure problems",
    "traceRegex": ".*RuntimeException.*",
    "matchedStatuses": [ "broken", "failed" ]
  },
  {
    "name": "Outdated tests",
    "messageRegex": ".*FileNotFound.*",
    "matchedStatuses": [
      "broken"
    ]
  }
]
just-boris commented 7 years ago

Hello! I am currently doing some work about migration to Allure 2 API. It will take some time, for the time being, you can provide categories.json yourself put it to allure-results folder manually.

doppelganger23 commented 7 years ago

Thanks! About broken statuses - I've done some research and figured out that statuses could not be controlled through categories.json (you can only change category for certain status). In fact, broken statuses are only available if you are using java runner which throws Exceptions, not Errors. Js test runners (for example Jasmine+Protractor) allways handle that exceptions and throw Errors, so you couldn't get broken test result via allure-js-commons anyway. Please fix me if I am wrong.

just-boris commented 7 years ago

The current package allure-js-commons doesn't do any assumptions about statuses. It receives testcase status via allure.endCase(status).

If you have problems with unexpected statuses, please report them to framework adapter that you are using. Currently we officially support two of them:

Also, a code sample that we can run and see ourselves is necessary for a good bug report.

doppelganger23 commented 7 years ago

Thanx again! I have raised an issue on allure-jasmine https://github.com/allure-framework/allure-jasmine/issues/32

wluu commented 5 years ago

Hello. Just curious, is allure-js-commons going to be replaced with https://github.com/allure-framework/allure-js?

baev commented 5 years ago

As soon as we'll find the person who would like to maintain it. Or later this year just after first Allure 3 milestone will be released