Move the counter out of the History object and into the formatter.

[schwernbot]

From: @schwern Date: Monday Sep 13, 2010 at 03:30 GMT Orig: https://github.com/Test-More/test-more/issues/71

The test counter is not a universal concept and doesn't belong in the History object. It also complicates the history by having to worry about ordering things. It doesn't belong in the builder either, they don't care what number the result is. Its really more an aspect of the formatter.

Move the Counter to the TAP Formatter. Formatters should not change the Result, so it shouldn't set $result->test_number, but it should respect it if its set. The builders should have a way to effect the counter, mostly to jump ahead in the case a totally unrelated system is outputting TAP.

[schwernbot]

From: @notbenh Date: Monday Sep 13, 2010 at 20:02 GMT Orig: https://github.com/Test-More/test-more/issues/71#issuecomment-404573

half done: http://github.com/schwern/test-more/blob/TB2-history-benh/lib/Test/Builder2/HistoryStack.pm

A counter free History based on the new Stack object. Currently it stores results and keeps stats. I've not played with the formatter but you could just use the id from the array to get the test number in the case that the result does not have test_number defined. Though this could get complicated if you have a mix of results with and without a test_number.

For example:

---
- test_number : 
- test_number : 14
- test_number : 
- test_number : 
- test_number : 1

Idealy that should produce:

ok 1
ok 2
ok 3
ok 4
ok 14

[schwernbot]

From: @schwern Date: Monday Sep 13, 2010 at 22:03 GMT Orig: https://github.com/Test-More/test-more/issues/71#issuecomment-404860

If the builder deliberately sets test_number, it wants to override the order for whatever reason. The TAP formatter should not try to put them back into the correct order.

The use case is where the builder outputs 1, 2 and 3 and then something totally unrelated outputs 4, 5 and 6 not using the formatter. The builder then sets its next result to be test_number 7 to cause the formatter to jump the count ahead.

Another use case is deliberately testing ordered parallel behavior. If you want to ensure that thread X terminates before thread Y you might tell thread X to output result 1 and thread Y to output result 2. If they're the wrong order you want that reflected in the output.

I see two possible outcomes for out of order tests. The first is if a result with its own test_result does nothing to the counter.

ok 1
ok 14
ok 2
ok 3
ok 1

The second is if the formatter sets the counter when it sees a test_number.

ok 1
ok 14
ok 15
ok 16
ok 1

I prefer the former. Less implied tied behavior. You can always change the formatter's counter if you need to.

In the end, there's two ways to alter the Formatter's count. The first is a one off change by setting Result->test_number. That just inserts an out of order result into the stream. The second is by setting Formatter->counter which changes the Formatter's count for all following results. I think that covers all use cases.

[schwernbot]

From: @notbenh Date: Monday Sep 13, 2010 at 23:05 GMT Orig: https://github.com/Test-More/test-more/issues/71#issuecomment-404982

I also prefer the 'former', it's more straight forward for the user and the developer:

for my $result ( @results ) {
   $self->out( sprintf q{%sok %d}, $result->is_fail ? 'not' : '',  $result->test_number || $counter++); 
}

[schwernbot]

From: @schwern Date: Thursday Oct 07, 2010 at 19:43 GMT Orig: https://github.com/Test-More/test-more/issues/71#issuecomment-454731

Solved by notbenh's recently merged TB2-history-notbenh