search

serenity-bdd / serenity-core

Serenity BDD is a test automation library designed to make writing automated acceptance tests easier, and more fun.
http://serenity-bdd.info
Other
721 stars 517 forks source link

Issues while running multiple CucumberWithSerenity runners in parallel #2565

Closed pogodins closed 2 years ago

pogodins commented 3 years ago

Since Cucumber with Junit can run only feature files in parallel I was looking for a solution to generate test runners for every scenario. Test runners generation intself works fine with https://github.com/temyers/cucumber-jvm-parallel-plugin.

However, in case of multiple runners being executed in parallel, I'm getting issues with Serenity, namely:

@wakaleo could you please check if I configured properly test execution? If I did it right, it would be really helpful to fix these issues as a lot of people would benefit from running scenarios in parallel.

When I'm able to execute multiple runners in parallel, I could write an insrtuction on how to generate runners and configure Serenity.

wakaleo commented 3 years ago

https://github.com/temyers/cucumber-jvm-parallel-plugin is no longer supported.

I've written up how to run Cucumber features in parallel in https://johnfergusonsmart.com/parallel-test-execution-with-cucumber-and-serenity-bdd/ - parallel scenario execution isn't supported.

pogodins commented 3 years ago

Hi @wakaleo, thanks for your feedback! My issue is not about cucumber plugin itself, and I know how to run feature files in parallel.

The problem for me is that Serenity doesn't work with parallel CucumberWithSerenity runners as I showed in my example project. In example projects I created runners pointing to different scenarios manually (without any plugin). What I expect is that manually created runners can be executed in parallel on the same machine.

Could you please suggest if I misconfigured my code or how we can fix Serenity to execute Cucumber runners in parallel?

globalworming commented 3 years ago

Last time i tried it seemed to work fine with multiple runners https://github.com/globalworming/serenity-cucumber-starter/commit/875c3265e0f7ac7d10f8ca3cba4cb7d494928e62#diff-7fa6615a67c71eb463e49be76cbe914bcc440e85b4eec68ef80e3ae9b16ad967

wakaleo commented 2 years ago

This seems to work fine in 3.2.0 - please reopen if the issue still occurs for you.

pogodins commented 2 years ago

The behavior is still the same for multiple runners with Serenity 3.2.0. Repository with example code is updated by me now. @wakaleo could you please share the configuration which worked for you?

wakaleo commented 2 years ago

The behavior is still the same for multiple runners with Serenity 3.2.0. Repository with example code is updated by me now. @wakaleo could you please share the configuration which worked for you?

See https://github.com/serenity-dojo/swaglabs-serenity-cucumber.git for an example of multiple Cucumber runners running in parallel with Serenity 3.2.0