Wrong step order in allure-report

IlgizVafin commented 7 years ago

[//]: # ( . Note: for support questions, please use Stackoverflow or Gitter. . This repository's issues are reserved for feature requests and bug reports. . . In case of any problems with Allure Jenkins plugin please use the following repository . to create an issue: https://github.com/jenkinsci/allure-plugin/issues . . Make sure you have a clear name for your issue. The name should start with a capital . letter and no dot is required in the end of the sentence. An example of good issue names: . . - The report is broken in IE11 . - Add an ability to disable default plugins . - Support emoji in test descriptions )

I'm submitting a ...

[x] bug report
[ ] feature request
[ ] support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

Wrong step order in allure-report.

Steps to reproduce:

create demo project for UI-testing based on framework webdriverio with Cucumber. You can use:
- boilerplate project - cucumber-boilerplate
- my demo project - https://github.com/IlgizVafin/webdriverio-demo-project2;

write some test scenario by Gherkin:


Feature: Broken feature

@Broken
Scenario: Go to github.com and press non-existent button
    When I go to https://github.com
    And I press button "Non-existent button"
    Then I expect page title "Join GitHub · GitHub"

Feature: Passed feature

@Passed
Scenario: Go to github.com and press signup button
    When I go to https://github.com
    And I press button "Sign up for GitHub"
    Then I expect page title "Join GitHub · GitHub"


3. run test - `wdio wdio.conf.js` (for example)
4. generate report - `allure generate ./allure-results` (for example)
5. open report and look at the step order
![image](https://user-images.githubusercontent.com/7014659/29922967-2ff505f6-8e71-11e7-9866-4326a27e6564.png)
6. step order isn't correct.

#### What is the expected behavior?
Step order should be equal to order in `*.feature` scenarious.

#### My environment:

| Allure version | 2.3.4 |
| --- | --- |
| Test framework | [webdriver.io](http://webdriver.io/) + cucumber@~2.2.0 |
| Generate report using | [allure-commandline@2.3.4](https://github.com/allure-framework/allure-npm) |

#### Other information 

[//]: # (
. e.g. detailed explanation, stacktraces, related issues, suggestions 
. how to fix, links for us to have more context, eg. Stackoverflow, Gitter etc
)

<!-- Love allure-report? Please consider supporting our collective:
👉  https://opencollective.com/allure-report/donate -->

slipwalker commented 7 years ago

I faced the same issue. Tests execution order is different from that Allure report shows (looks like tests order in Allure report has some random nature). Even, when for every test explicitly set priority on a test level (in case using TestNG framework for instance). I have simple test class with test methods and they have different order in Allure that they were actually executed.

eliranshani commented 7 years ago

Can you share when this fix will be released ?

lrocheWB commented 7 years ago

+1

maciejkamela commented 7 years ago

Have the same issue. Moreover when I run eg. 8 scenarios they are mismatched and just 5 was presented but in wrong order. I would expect the test outcome to be grouped by feature.

lrocheWB commented 7 years ago

@baev thanks ! https://github.com/allure-framework/allure2/releases/tag/2.3.5

It works for me.

ravxz commented 6 years ago

Have similar issue with TestNG DataProvider tests. The test cases under test class in report are ordered backward like: #3, #2, #1. Meanwhile, default shown order is by "name" forward.