Support parallel execution of several ETSs in one test run

[michellutz]

Background and Motivation:

Performance optimisation is required to reduce startup time and validation time; reduction of startup time will simplify cloud deployment horizontal scaling, while reduction in validation time will be helpful while integrating ETF with INSPIRE Geoportal, or any other Metadata related workflow/pipeline.

One identified performance issue is that each Executable Test Suite is executed sequentially, not in parallel.

Proposed change

If a Test Run contains multiple ETSs, run them in parallel (where possible).

It still remains to clarify how dependencies are treated - just ignore dependencies and start all ETS or wait until dependant ETSs have finished.

It could also be important, whether the executed ETSs are a group of service tests that put the service in different states.

Starting and managing the threads would be done by ETF.

Alternatives

n/a

Funding

JRC will be ready to fund within its current development contract.

Additional information

n/a

[michellutz]

Split off from #14 as agreed in the 4th SG meeting on 2018-09-04.

[jenriquesoriano]

To do:

• Analysis of real tests to assess if in practical terms it is possible to parallelize the execution. Analyze for which cases it is possible to go parallel or not.

[carlospzurita]

After reviewing the TestTasks for BaseX or SoapUI tests, it seems like each of them can be run indepently. They don't share environment variables and they their own temporary files on the folder structure of the ETF. We think that this could be feasible