The ESR will get information about sub suites to start via the Environmen Defined events from the environment provider. These are then executed and the ESR will wait for test results. However no verification of the fact that all expected sub suites started and finished is done.
Motivation
We should always verify that we started everything that was expected and not accidentally hide any information from the users.
Exemplification
A person starts ETOS tests, the ESR asks the environment provider for an environment, the environment provider splits the suite and sends Environment Defined. The ESR starts up the sub suites from these Environment Defined events, but misses one.
In this case the user will get an "OK" back that all tests ran, but all tests did not execute.
Benefits
Clearer failures when something really bad happens.
Possible Drawbacks
This adds complexity and the information from the environment provider and the Environment Defined events must be equal.
Description
The ESR will get information about sub suites to start via the Environmen Defined events from the environment provider. These are then executed and the ESR will wait for test results. However no verification of the fact that all expected sub suites started and finished is done.
Motivation
We should always verify that we started everything that was expected and not accidentally hide any information from the users.
Exemplification
A person starts ETOS tests, the ESR asks the environment provider for an environment, the environment provider splits the suite and sends Environment Defined. The ESR starts up the sub suites from these Environment Defined events, but misses one. In this case the user will get an "OK" back that all tests ran, but all tests did not execute.
Benefits
Clearer failures when something really bad happens.
Possible Drawbacks
This adds complexity and the information from the environment provider and the Environment Defined events must be equal.