Closed DJ-Glock closed 9 months ago
For Jupiter, the closest thing you could do would be to run test classes in parallel but methods in each class sequentially as documented in the User Guide via:
junit.jupiter.execution.parallel.mode.default = same_thread
junit.jupiter.execution.parallel.mode.classes.default = concurrent
I took a quick look at the Cucumber test engine and it doesn't seem to provide equivalent options. Could you please open an issue in the cucumber-jvm repo instead?
@marcphilipp well, it works this way with same_thread
value.
With cucumber.execution.execution-mode.feature=same_thread
- it works as you described, test classes run in parallel, methods in test class run sequetially.
With cucumber.execution.execution-mode.feature= concurrent
- all methods will run in parallel.
But I would like to have a combined behaviour.
For Jupiter, the closest thing you could do would be to run test classes in parallel but methods in each class sequentially as documented in the User Guide via:
junit.jupiter.execution.parallel.mode.default = same_thread junit.jupiter.execution.parallel.mode.classes.default = concurrent
I took a quick look at the Cucumber test engine and it doesn't seem to provide equivalent options. Could you please open an issue in the cucumber-jvm repo instead?
Hi @marcphilipp Can you please check my last previous comment? I believe you closed it wrongly.
As pointed out by @mpkorstanje, limited concurrent execution within one test class/feature file is currently not possible. Would it help if you implemented that behind the scenes when accessing the services? For example, you could add a decorator around your service that would block calls when the limit has been reached.
Ah, unfortunately adding decorators like this is not what I am looking for. Application is safe, it won't process more than it could. The issue is in tests - I want them to be stable, efficient and with predictable execution time. If I go this way, I'll need to increase timeouts a lot and it will be hard to predict execution time for each particular test. It will be a mess.
If you would like us to be able to process this issue, please provide the requested information. If the information is not provided within the next 3 weeks, we will be unable to proceed and this issue will be closed.
Closing due to lack of requested feedback. If you would like to proceed with your contribution, please provide the requested information and we will re-open this issue.
Discussed in https://github.com/junit-team/junit5/discussions/3513
Originally posted by DJ-Glock October 23, 2023
I use cucumber with JUnit.
I have several feature files. Each feature file is designed to test one application instance: A.feature, B.feature, C.feature. Each feature file contains one scenario template(outline) with N examples (usually more than 10).
Application instances are independent from each other. And each application instance can process input data in M threads, usually 2. So it does not make sense to run scenarios for particular application instance in more than M (2) threads, because it will lead to flaky tests and unstable execution time.
In the same time running scenarios for one instance (with execution-mode.feature=same_thread) in one thread can take much time.
What do I want to achieve:
Example:
Being executed simultaneously:
Waiting in the queue:
Graphical visualization for better understanding:
I suppose that it may be achieved using
CustomStrategy
: https://github.com/cucumber/cucumber-jvm/tree/main/cucumber-junit-platform-engine#parallel-executionBut it could be tricky especially with the fact that I do not understand for now, at what point of time these properties are being set. And could they be changed dynamically (another words based on feature files configuration, not only
pom.xml
).My idea was to try using tags on scenario level, something like @feature=A, @feature=B etc to limit parallelism for pool of scenarios. But I'm on the beginning of my journey, so need an advice if it's possible or not.
Question on SO: https://stackoverflow.com/questions/77388756/configure-parallelism-on-features-and-scenarios-level-in-the-same-time-cucumbe
After discussion on SO with @mpkorstanje, he advised that with current implementation of JUnit 5 it's not possible to do it, quote:
I also tried another approach: tried to set up several Suites/TestRunners (one test runner per feature(s) I want to run per app instance) with configured parallelism for each runner, for example:
But it does not met my requirements because test runners are run sequentially.
I'm not very experienced in java programming, so if someone from the community is willing to try implementing this mechanism, it would be great.