search

hrcorval / behavex

BDD test wrapper for parallel test executions and more!
https://github.com/hrcorval/behavex
MIT License
85 stars 20 forks source link

Does not work with scenario outline #28

Closed apetrenko-modus closed 2 years ago

apetrenko-modus commented 2 years ago

Scenarios which were implemented as scenario outline were not executed. Their status in report is Automated Not Run

hrcorval commented 2 years ago

We did a fix in version 1.5.9. Please let us know if it doesn't work with that version and we will provide a fix asap. We are going to provide all necessary fixes to have a stable version

hrcorval commented 2 years ago

Issue already fixed and tested. You can try with version >=1.5.9

apetrenko-modus commented 2 years ago

Verified on 1.5.9. Not fixed. Still ignores Scenario outlines

image

If I split this scenario outline on scenarios then behavex executes them @hrcorval

hrcorval commented 2 years ago

It seems there is something else we are not considering. Is it possible to share the tags you are using? If you can share the feature file without the implementation files, removing any sensitive information, it will help us to reproduce and fix it. Thanks!!

apetrenko-modus commented 2 years ago

Ok. will prepare

venkat1993vcb commented 2 years ago

Yes , as said by @apetrenko-modus unable to run parallely for Scenario Outline.

hrcorval commented 2 years ago

Thanks @venkat1993vcb for reporting it. Do you have a sample scenario that can help me on reproducing this issue?. Thanks!

venkat1993vcb commented 2 years ago

Normally the scenario outlines are running in parallel, but I'm trying to run two scenario outlines parallelly by adding values to the example table dynamically. One scenario outline will 116 scenarios , also i think the scenarios are running but the report is not generating properly. Please find the below image for reference. When checking the temp result files found that the test status is skipped for all the scenarios.

image

hrcorval commented 2 years ago

Hi @venkat1993vcb. I have been trying to reproduce the bug by dynamically adding examples to the table, but it seems the mechanism I'm using is not the same as in your implementation. TO be able to reproduce the error accordingly, can you please share a sample implementation on how you do to add examples to the table at runtime? Thanks!

hrcorval commented 2 years ago

Hi @apetrenko-modus , I'm just wondering if you have been able to deal with the scenario outline that is not properly reported. Also, I just wanted to know if the scenario have any special implementation that I should be aware of , to be able to reproduce it. Thanks

writetomaha commented 2 years ago

I am also facing this issue, unable to run the parallel execution for scenario outline.

lazareviczoran commented 1 year ago

Hi!

We also ran into this issue as we added the some scenario outlines. We are using version 1.6.0. The way we ran the tests is by running behavex --parallel-process 2 command, so it was using the default parallel-scheme (which is scenario). That way the scenario outlines end up being skipped.

On the other hand, we noticed that when we tried to run behavex --parallel-process 2 --parallel-scheme feature all the scenario outlines have ran successfully, but this is not an ideal workaround since our feature files contain multiple scenarios that will run sequentually which can slow down the overall execution time, and we would want to have the scenario outlines running in parallel.

hrcorval commented 3 weeks ago

The parallel execution of scenario outlines has been fixed in latest versions, and this is a feature we are exhaustively using. We apologize for not getting this feature properly running in versions stated above. Latest version is working as expected. Hope it helps!