Closed Art4 closed 4 months ago
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 98.54%. Comparing base (
dfa6b49
) to head (e502824
).
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Using Behat for BDD and BDT is really impressive and useful. For new features I create a temporary tag
@wip
and rundocker compose exec php composer behat -- --tags=wip
. To test the feature on a specific redmine version I rundocker compose exec php composer behat -- --tags=issue,group --suite=redmine=50103
. If the feature is ready I run all behat tests on all redmine versions usingdocker compose exec php composer behat
.However the tests are becoming bigger and that leads to 1.) higher usage of time and memory to execute all tests at once. And 2.) it is not possible to change the format of the output using CLI (because the
--output=progress
is already declared incomposer.json
).This PR solves these problems by creating a new
bdt
composer command for tests grouped by Redmine versions (aka behat suites). The output there is always formatted asprogress
. If one test fails it is now easier to find the Redmine version, where the test has failed thanks to the separation by behat suite.Test-run: docker compose exec php composer bdt
```bash $ docker compose exec php composer bdt > behat --config tests/Behat/behat.yml '--format=progress' '--suite=redmine_50103' ...................................................................... 70 ...................................................................... 140 ...................................................................... 210 ...................................................................... 280 ...................................................................... 350 ...................................................................... 420 ...................................................................... 490 ...................................................................... 560 ...................................................................... 630 ...................................................................... 700 ...................................................................... 770 ...................................................................... 840 .............................................. 89 scenarios (89 passed) 886 steps (886 passed) 1m56.30s (15.69Mb) > behat --config tests/Behat/behat.yml '--format=progress' '--suite=redmine_50009' ...................................................................... 70 ...................................................................... 140 ...................................................................... 210 ...................................................................... 280 ...................................................................... 350 ...................................................................... 420 ...................................................................... 490 ...................................................................... 560 ...................................................................... 630 ...................................................................... 700 ...................................................................... 770 ...................................................................... 840 .............................................. 89 scenarios (89 passed) 886 steps (886 passed) 2m9.03s (15.66Mb) > behat --config tests/Behat/behat.yml '--format=progress' '--suite=redmine_40210' ...................................................................... 70 ...................................................................... 140 ...................................................................... 210 ...................................................................... 280 ...................................................................... 350 ...................................................................... 420 ...................................................................... 490 ...................................................................... 560 ...................................................................... 630 ...................................................................... 700 ...................................................................... 770 ...................................................................... 840 .. 84 scenarios (84 passed) 842 steps (842 passed) 1m59.39s (15.57Mb) ```The
behat
command will be changed to have no output format (defaults to--format=pretty
). So if I rundocker compose exec php composer behat -- --tags=wip
I will get the pretty format that helps me with debugging. If I want to have theprogress
output I can declare it withdocker compose exec php composer behat -- --tags=wip --format=progress
.This PR also improves the test docs explaining the different commands. I've tested theses new commands while working on #419.