The only times libtest runs tests on the main thread are
The platform doesn't support threads
We hit the max number of threads
Its a benchmark
On the contrary, libtest-mimic prefers running tests on the main thread, including if there is only one test case. I went ahead with the libtest-mimic approach.
As for threads, I went ahead and create/destroy them one per-test, like libtest, just to get up and running. No idea if it'd be faster to use a thread-pool like libtest-mimic.
For simplicity of communication, I made the tests communicate across threads by sending Events like normal test runs. This keeps the implementation simpler than libtest's approach of notifying test start on the main thread before spawning the thread. I also expect this will make things simpler for when we add fixture support.
I have not yet implemented timeout support. That is important because there isn't another way to know what the current thread is that is taking too long without re-running with --test-threads=1 which might not be practical when there are a lot of tests. On the other hand, a 60s timeout will likely lead people to hit ctrl-c for a small number of tests, never knowing a timeout exists. imo we are conflating UI and test interactions. What I'd like to see is the UI provide progress bars and report what the tests taking too long are while we also have test timeout error checks as a layer sitting above tests.
The only times libtest runs tests on the main thread are
On the contrary, libtest-mimic prefers running tests on the main thread, including if there is only one test case. I went ahead with the libtest-mimic approach.
As for threads, I went ahead and create/destroy them one per-test, like libtest, just to get up and running. No idea if it'd be faster to use a thread-pool like libtest-mimic.
For simplicity of communication, I made the tests communicate across threads by sending
Events like normal test runs. This keeps the implementation simpler than libtest's approach of notifying test start on the main thread before spawning the thread. I also expect this will make things simpler for when we add fixture support.I have not yet implemented timeout support. That is important because there isn't another way to know what the current thread is that is taking too long without re-running with
--test-threads=1which might not be practical when there are a lot of tests. On the other hand, a 60s timeout will likely lead people to hit ctrl-c for a small number of tests, never knowing a timeout exists. imo we are conflating UI and test interactions. What I'd like to see is the UI provide progress bars and report what the tests taking too long are while we also have test timeout error checks as a layer sitting above tests.Fixes #9