Typically I expect running tests to just show what test is being run, and then when the tests have been run, to see captured output associated with a failure if there is a failure.
With our test setup though, we get output streamed and everything becomes garbled. For example I know I'm not the only one to have been confused about seeing an error about not starting as PID 1 - but that was just a test that verified that we error nicely when starting as something else than PID 1 etc.
Here is another example where two tests pass run and we see the logs from that streamed. The first test is running something expected to fail - which it does (XFAIL), and one that is expected to succeed - which it does (PASSED). Skimming through this, one would think something is wrong - but its not, so its just a distraction.
I expected this to look more like this:
Suggested action
[x] Let's capture relevant logs while a test is running, and then emit it on failure like typically done.
[x] Let's configure pytest with --max-fail=1 to quicker see information about a failed tests, and make debugging it easier. We won't see all failures, but we may improve our ability to understand the first failure we encounter and get the captured logs presented faster.
Typically I expect running tests to just show what test is being run, and then when the tests have been run, to see captured output associated with a failure if there is a failure.
With our test setup though, we get output streamed and everything becomes garbled. For example I know I'm not the only one to have been confused about seeing an error about not starting as PID 1 - but that was just a test that verified that we error nicely when starting as something else than PID 1 etc.
Here is another example where two tests pass run and we see the logs from that streamed. The first test is running something expected to fail - which it does (XFAIL), and one that is expected to succeed - which it does (PASSED). Skimming through this, one would think something is wrong - but its not, so its just a distraction.
I expected this to look more like this:
Suggested action
--max-fail=1to quicker see information about a failed tests, and make debugging it easier. We won't see all failures, but we may improve our ability to understand the first failure we encounter and get the captured logs presented faster.