Closed paulmillar closed 2 years ago
I typically add a dbg
label to a test and set ROBOT_ARGS="-i dbg"
, since I don't like using the test name as it requires quoting.
You could use ROBOT_ARGS="--name \"test name\""
as well, but probably the approach you propose is simpler for the user.
Hiya,
The dbg
label is a good hint, but has the (minor) drawback of requiring one to edit the test suide.
As you say, one might expect to be able to use ROBOT_ARGS
.
I started off going down that line, attempting with commands like:
ROBOT_ARGS="-t \"test name\"" run-testsuite.sh
It took an embarrassingly long time to figure out why this doesn't work (hint: eval
) -- partly from a somewhat cryptic error message and partly because it works perfectly fine with running the robot
command on the command-line.
Since getting ROBOT_ARGS
environment variable to work with quoted arguments required patching the run-testsuite.sh
script, I thought I'd add something a bit nicer: using ROBOT_ARGS
is a faff after all.
Some time ago I did something similar for the VOMS testsuite. The most relevant commit is https://github.com/italiangrid/voms-testsuite/commit/91b8a5cd0aabe14a1170f352c074b989f3a7c7e9. It would be a more general solution for passing arguments from the command line of run-testsuite.sh
down to robot
. What do you think?
Hi all.
@giacomini That would work for me, too.
I'd have thought running a single test is a sufficiently common desire that having it as a special case would make sense; however, I'm happy with whichever solution allows me to run a single test.
Aside: arrays are (IIRC) definitely a bash-ism, so taking the script out of the realm of POSIX shell. I have no problem with this, just mentioning it just in case it matters.
Cheers, Paul.
I have just created PR #24. Would you please have a look?
For example, now one can do:
[test@d9dffc4ffe20 test-suite]$ ./run-testsuite.sh se-cnaf-amnesiac-storm --test "Token with correct *"
JWT compliance test suite run against: se-cnaf-amnesiac-storm
==============================================================================
se-cnaf-amnesiac-storm
==============================================================================
se-cnaf-amnesiac-storm.Audience
==============================================================================
Token with correct audience is accepted | PASS |
------------------------------------------------------------------------------
Token with correct audience in multiple option is accepted | PASS |
------------------------------------------------------------------------------
se-cnaf-amnesiac-storm.Audience | PASS |
2 tests, 2 passed, 0 failed
==============================================================================
se-cnaf-amnesiac-storm | PASS |
2 tests, 2 passed, 0 failed
==============================================================================
Output: /home/test/test-suite/reports/se-cnaf-amnesiac-storm/output.xml
Log: /home/test/test-suite/reports/se-cnaf-amnesiac-storm/log.html
Report: /home/test/test-suite/reports/se-cnaf-amnesiac-storm/report.html
Motivation:
Sometimes, typically when debugging a specific problem, it is useful to run just a single test from the test-suite. This allows for more extensive debugging to be enabled.
Modification:
Add an optional second command-line argument to 'run-testsuite.sh' that limits the testing to a specific test.
When only a single command-line argument is provided then the tests are run as before.
Result:
It is now easy to run a single test from the suite.