diagnostics/comments only accepted in subtests if indented

[robertlemmen]

prove6 only accepts comments/diagnostics in subtests if they are indented correctly, as shown by this small examples:

use Test;

ok(True, "A");
say "# test A";
subtest {
    plan 2;
    is(1, 1, "1");
    say "# test B";
    is(2, 2, "2"); 
}
ok(!False, "B");
done-testing;

My reading of the TAP "spec" is that comments should be ignored independently of their indentation, that is also the behavior of perl5 prove. there is also a practical reason to allow this: it allows diagnostics from somewhere deeper in the code being tested, that should not rely on Test.pm6 itself, and should certainly not need to be aware of the level of subtest it is being called from.

perl5 sample:

use Test::More;

ok(1);
print "# A\n";
subtest try => sub {
    plan tests => 1;
    print "# A\n";
    ok(1);
};
done_testing;

I am happy top provide a patch, but need some steer on the direction to go in. My instinctive reaction would be to remove the comment from the grammar, and just have a simple match in the loop that goes over the lines in the input...

[ufobat]

Thats the Same issue that caused ihr Troubles in #21

[Leont]

My reading of the TAP "spec" is that comments should be ignored independently of their indentation

Which spec? AFAIK there have been several such attempts but subtests were mostly absent.

that is also the behavior of perl5 prove.

Prove5 ignores subtests entirely. Very few harnesses parse them for that matter, which is the main reason why this area is so poorly described. If anything this behavior is described by "what do other harnesses do".

there is also a practical reason to allow this: it allows diagnostics from somewhere deeper in the code being tested, that should not rely on Test.pm6 itself, and should certainly not need to be aware of the level of subtest it is being called from.

I'm genuinely puzzled you would want this. Mainly because outputting the wrong thing (such as a sentence starting with the word "ok", a sentence starting with "1.." or the sentence "Bail out!") can royally mess up your test output. Likewise one might trigger a future extension to TAP. And even if you know how to output TAP correctly, a future version of your test framework might output something different (e.g. one where # doesn't start a comment).

Why would you do this to yourself? Test frameworks exist to abstract all this pain.

[ufobat]

In oder to get the test fixed you could use note.

use Test;

ok(True, "A");
note "test A";
subtest {
    plan 2;
    is(1, 1, "1");
    note "test B";
    is(2, 2, "2");
}
ok(!False, "B");
done-testing;

For me personally this is more a LTA, because what you need to see is something like "Tap format invalid" because one is probably not aware of that.

In addition it might be useful if you could "silence" the regular output of the code you're testing.

[Leont]

I went digging into the last spec draft, and it seems to suggest the current behavior ir correct

• A comment line with the name of the subtest: /^ {4}# Subtest: (.*)$/
• A TAP14 stream, indented 4 spaces.
• A (non-indented) test point indicating the success or failure of the subtest, with a description matching the Subtest's name.

In that light I'm closing this ticket.