Most environments, like Maven plugin development, Springboot, you name it, rely on being able to "fake" enough of the underlying system to run tests. Or provide more info than the production build would see, and so on.
One example of what you may want to do with a test container, is to use an injected System environment variable getter. It turns out that there is lots of reusable functionality that can be applied if an app can just know such as simple thing that it is actually running from inside a CI/CD pipeline, enad so on.
Brainstorming this further, it would also be useful to do things like launch a process in the background, or a Thread, or some other logic that runs alongside the module being executed. I suspect this is something that resides in the test harness container, and apps use a well defined API to ask them questions or do things like that. For example, if I could "inject" a Java thread that polls the XTC runtime every X seconds and dumps stack traces, by just sending SIGQUIT to the javatools launcher/Runner, run an arbitrary JCMD, it would help debugging things like deadlocks a lot.
(We should also look into other ways to detect anomalous behavior, for example just running JFR with or without deadlock detection as part of the CI/CD build starts to feel like a good idea right now.)
Any test container would likely also integrate well with XUnit. Perhaps it can inject the configured test suite of XTC modules and their expected outputs/results into the build?
But let's not get ahead of ourselves. I believe that a great POC to figure out how a model like this for testing should work, is to implement the injection of environment variables into the XTC build. That would make it very simple to set up more automated test logic on the GitHub workflow side, but it's by no means constrained to GitHub workflows.
Most environments, like Maven plugin development, Springboot, you name it, rely on being able to "fake" enough of the underlying system to run tests. Or provide more info than the production build would see, and so on.
One example of what you may want to do with a test container, is to use an injected System environment variable getter. It turns out that there is lots of reusable functionality that can be applied if an app can just know such as simple thing that it is actually running from inside a CI/CD pipeline, enad so on.
Brainstorming this further, it would also be useful to do things like launch a process in the background, or a Thread, or some other logic that runs alongside the module being executed. I suspect this is something that resides in the test harness container, and apps use a well defined API to ask them questions or do things like that. For example, if I could "inject" a Java thread that polls the XTC runtime every X seconds and dumps stack traces, by just sending SIGQUIT to the javatools launcher/Runner, run an arbitrary JCMD, it would help debugging things like deadlocks a lot.
(We should also look into other ways to detect anomalous behavior, for example just running JFR with or without deadlock detection as part of the CI/CD build starts to feel like a good idea right now.)
Any test container would likely also integrate well with XUnit. Perhaps it can inject the configured test suite of XTC modules and their expected outputs/results into the build?
But let's not get ahead of ourselves. I believe that a great POC to figure out how a model like this for testing should work, is to implement the injection of environment variables into the XTC build. That would make it very simple to set up more automated test logic on the GitHub workflow side, but it's by no means constrained to GitHub workflows.