[tester] an an option to fork a new framework per test or class

[laeubi]

As mentioned here https://github.com/bndtools/bnd/issues/5509#issuecomment-1382107436 currently the isolation of tests might be not appropriate enough.

In pax exam there is a configuration option to control how often a container is started/shut down:

1. per suite: all discovered test-classes and tests share the same container, this is the fastest, but also the less isolated option
2. per class: all tests of a testclass, this is moderately fast and isolated per class
3. per test: each test is executed in an own container, this is the slowest but highest isolation level

Currently bnd tester uses the per suite strategy, but already supports per class if one specify a single test-class name

it would we good to support the same here , e.g with tester.isolation = suite, class, test, where depending on the strategy the container is started/stopped accordingly. Given that the OSGi-Framework itself actually starts quite fast and most of the hard work (setup and resolve) must only performed once, this could actually be not much slower than the current strategy, together with

• https://github.com/bndtools/bnd/issues/5509

this could even be faster than today given that multi-cpu systems are standard today, so with class or test combined with fork many tests can run and complete even faster than a single run and have better isolation.

[laeubi]

Only challenge with test strategy is that one needs to sync the update of the result file (e.g using file-level locking mechanism) when using a forked mode.

[laeubi]

Probably a first step would be to support a syntax similar to maven-surefire to execute a single test method: https://maven.apache.org/surefire/maven-surefire-plugin/examples/single-test.html#running-a-set-of-methods-in-a-single-test-class

for example test=TestCircle#mytest

[kriegfrj]

Although I can see the usefulness of this suggestion, I'm not sure how feasible it is with the current architecture.

The Bnd JUnit Platform tester is essentially a thin OSGi wrapper around JUnit Platform, and it is a bundle (ie, it is designed to be run inside a running framework). The JUnit Platform Launcher library is responsible for coordinating the discovery and launching of tests. It does not have any support for forking new processes.

Going one level down, the Jupiter engine (which supports parallelism) also doesn't support forking new processes, only new threads.

So it will be very tricky to get this working. It would require either significant changes to Jupiter itself, or else to almost completely take over the discovery process.

[pkriens]

Not likely to be implemented