Support for test hierarchy

jonhoo commented 9 years ago

It would be useful to have the ability to further structure the tests beyond just single tests. For example, if each test file could be given a "suite" title, we could get test output like:

 ✓ root directory layout
     ✓ contains .
     ✓ contains ..
     ✓ contains etc
 ✓ root directory manipulation
     ✓ can add file as root
     ✓ cannot add file as user
 ✓ nested directories
     ✓ can create directory in / as root
     ✓ can create files in subdirectory as root

Personally I'd like to be able to nest tests further (maybe by being able to load a test file with sub-tests inside a @test), but even a single level of nesting would be useful.

esiegerman commented 8 years ago

I've achieved something like this result with a naming scheme for tests:

@test "root layout: contains ." { ... }
@test "root layout: contains .." { ... }
@test "root layout: contains etc" { ... }
@test "root manipulation: can add file as root" { ... }
@test "root manipulation: ..." { ... }

But yes, your way would be better -- easier both to write and to read.

bsideup commented 8 years ago

:+1:

harschware commented 8 years ago

The TAP format already has support for subtests. It is achieved by writing out a new test plan, and indenting it and the subtest results with four spaces.

I haven't been able to find it in the TAP13 spec yet, but at least Perl outputs it: https://metacpan.org/pod/Test::More#Module-tests and scroll up (or search for subtest) .. and Jenkins TAP plugin can consume it. https://wiki.jenkins-ci.org/display/JENKINS/TAP+Plugin

sstephenson / bats

Support for test hierarchy #130