Add support for "dynamic" default runner selection via system property

[k1w1m8]

I have a use case where I want to use a custom Runner for my test classes without forcing them to each define a @RunWith.

One way to do that is to define a system property, eg junit.runner=<classname>, and then update the builder list in AllDefaultPossibilitiesBuilder#runnerForClass with a new SystemPropertyBuilder which optionally creates Runners based on a system property

Would JUnit accept a small PR for that?

[kcooney]

Generally we prefer that the behavior of the tests be specified via the code. Having the behavior change via system properties could result in tests behaving one way via a continuous build but a different way when run via an IDE.

[k1w1m8]

That is indeed the purpose here. I want to use a different runner in CI than on local.

On Thu, Feb 9, 2023, 2:05 PM Kevin Cooney @.***> wrote:

Generally we prefer that the behavior of the tests be specified via the code. Having the behavior change via system properties could result in tests behaving one way via a continuous build but a different way when run via an IDE.

— Reply to this email directly, view it on GitHub https://github.com/junit-team/junit4/issues/1759#issuecomment-1423556387, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACVH6EDNPAGFLJ2QVPSXUVTWWRNFLANCNFSM6AAAAAAUV6YS2Y . You are receiving this because you authored the thread.Message ID: @.***>

[kcooney]

Could you provide some details as to why you would want to use a different runner in CI? That would open up the possibly that developers couldn't reproduce CI failures when running locally, or could break the CI even though the test passed locally.

[k1w1m8]

The CI system has some technical limitations which means we want to run the tests inside a container on CI. Using a Runner that optionally runs the tests remotely supports that.

On Fri, Feb 10, 2023, 4:03 PM Kevin Cooney @.***> wrote:

Could you provide some details as to why you would want to use a different runner in CI? That would open up the possibly that developers couldn't reproduce CI failures when running locally, or could break the CI even though the test passed locally.

— Reply to this email directly, view it on GitHub https://github.com/junit-team/junit4/issues/1759#issuecomment-1425177404, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACVH6EBECXGQBKZYPCFTGE3WWXDYBANCNFSM6AAAAAAUV6YS2Y . You are receiving this because you authored the thread.Message ID: @.***>

[kcooney]

Thanks for the explanation.

The JUnit 4 maintainers have been reluctant to add features to where the behavior the test is not evident from the annotations on the test classes, methods and fields.

I suggest writing your own runner that enables this special behavior based on the value of an environment variable or system property, and updating your test classes to specify that runner.

Another thing to keep in mind: JUnit 4 is in maintenance mode, and we are not sure when there might be another release.

[kcooney]

It looks like this is related to https://github.com/datastax/remote-junit-runner

I suggest you reach out to the developers of that library and ask for a way to update the runner to optionally run the test locally if a given system property is set. No changes to JUnit would be needed.

[k1w1m8]

Fair enough. Thanks for the clarification.

On Sat, Feb 11, 2023, 3:24 PM Kevin Cooney @.***> wrote:

Thanks for the explanation.

The JUnit 4 maintainers have been reluctant to add features to where the behavior the test is not evident from the annotations on the test classes, methods and fields.

I suggest writing your own runner that enables this special behavior based on the value of an environment variable or system property, and updating your test classes to specify that runner.

Another thing to keep in mind: JUnit 4 is in maintenance mode, and we are not sure when there might be another release.

— Reply to this email directly, view it on GitHub https://github.com/junit-team/junit4/issues/1759#issuecomment-1426609411, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACVH6EDM3PA4W3XUX2MDHU3WW4H5VANCNFSM6AAAAAAUV6YS2Y . You are receiving this because you authored the thread.Message ID: @.***>

[k1w1m8]

Ahh, my last response was incorrect. Sorry. Let me try again.

I have tens of thousands of tests across many different repos which have no Runner annotation, they are using the default runner.

I now want to run those tests in exactly the same way, but in a remote JVM, but I don't want to have to invasively add thousands of identical new Runner annotations to do that.

Thats what the PR would be for. Basically just a few lines in JUnit4Builder which checks a system property and creates a BlockJUnit4ClassRunner via reflection if found.