runner: Test run output on console/terminal is not really useful

[jenisys]

Describe the bug The output of the parallel test run is not really useful. It contains:

• Diagnostics: Not really readable
• Unreadable JSON output of failures (on scenarios marked with @xfail ), not beautified
• Partial result summaries that can not be related to concrete features/scenarios, like:

...
Took 0m0.000s
1 feature passed, 0 failed, 0 skipped
6 scenarios passed, 0 failed, 2 skipped
0 steps passed, 0 failed, 0 skipped, 0 undefined
Took 0m0.000s
1 feature passed, 0 failed, 0 skipped
6 scenarios passed, 0 failed, 2 skipped
0 steps passed, 0 failed, 0 skipped, 0 undefined
Took 0m0.000s
1 feature passed, 0 failed, 0 skipped
6 scenarios passed, 0 failed, 2 skipped
0 steps passed, 0 failed, 0 skipped, 0 undefined
Took 0m0.000s
...

To Reproduce Steps to reproduce the behavior:

1. Run 'behavex' against the features/ directory of the behave repo

Expected behavior

• Show a progress-bar how many files/features/scenarios are processed (and which feature failed w/ how many scenarios)

EXAMPLE OUTPUT:

  -- STATE: In progress during test run by using a progress-bar
   25% <########...............> features/some.feature  (duration: 32s)

  -- HINT: On completed features of this test-run on files
  PASSED  features/good.feature  (6 scenarios: 6 passed, 0 failed)
  FAILED  features/bad_1.feature (5 scenarios: 4 passed, 1 failed)

   -- ALTERNATIVE: Both parts combined 
   --   Finished feature-file status overrides the progress-bar, and progress-bar uses new/next-line
  PASSED  features/good.feature  (6 scenarios: 6 passed, 0 failed)
  FAILED  features/bad_1.feature (5 scenarios: 4 passed, 1 failed)
   25% <########...............> features/some.feature  (duration: 32s)

... final summary in the end (when everything is finished; already provided)

• OPTIONAL: Use greed/red coloring on result on the progress-bar and completed results if possible
• Hide the garbage output that is unreadable
• OR: Make the garbage output readable

REASON:

• The test reports should contain this kind of output already

Version Info:

• behavex v2.0.2 (current HEAD of repository)

[etherfurnace]

👍

[hrcorval]

Hi @jenisys, @etherfurnace , I agree with your comments, so I have added a progress bar that can be enabled using the --show-progress-bar argument when running the tests in parallel. This has been implemented in release version 3.2.13. Of course, this is the first progress bar version, and we will continue enhancing the implementation across releases, considering also as part of that the additional details you provided above. Details have been provided in documentation. Thanks!!