xp-framework / unittest

Unittests for the XP Framework
0 stars 0 forks source link

Add "stop after first failing test" option #11

Closed thekid closed 8 years ago

thekid commented 8 years ago

This pull request implement issue #3 and adds a facility for stopping the test run when a certain event occurs.

Usage

class ExampleTest extends \unittest\TestCase {

  #[@test]
  public function skipped() {
    $this->skip('Because I can');
  }

  #[@test]
  public function fails() {
    $this->fail('Because I do');
  }

  #[@test]
  public function succeeds() {
    // NOOP
  }
}

Regular run

$ unittest ExampleTest.class.php
[SF.]

F unittest.TestAssertionFailed(test= ExampleTest::fails, time= 0.000 seconds) {
  unittest.AssertionFailedError{ Because I do }
    at ExampleTest::fails() [line 0 of StackTraceElement.class.php]
# ...
 }

✗: 2/3 run (1 skipped), 1 succeeded, 1 failed
Memory used: 1414.89 kB (1469.10 kB peak)
Time taken: 0.000 seconds

With "stop on skip"

$ unittest -s skip ExampleTest.class.php
[S|

■ unittest.IgnoredBecause{ Because I can }: 0/1 run (1 skipped), 0 succeeded, 0 failed
Memory used: 1405.65 kB (1469.30 kB peak)
Time taken: 0.000 seconds

With "stop on fail"

$ unittest -s fail ExampleTest.class.php
[SF|

F unittest.TestAssertionFailed(test= ExampleTest::fails, time= 0.000 seconds) {
  unittest.AssertionFailedError{ Because I do }
    at ExampleTest::fails() [line 0 of StackTraceElement.class.php]
# ...
 }

■ unittest.AssertionFailedError{ Because I do }: 1/2 run (1 skipped), 0 succeeded, 1 failed
Memory used: 1424.54 kB (1470.46 kB peak)
Time taken: 0.000 seconds
thekid commented 8 years ago

Real-life usecase: Discovering ignored tests. Here, the test parameter_type_determined_via_scalar_syntax hadn't been un-ignored after PHP 7 gained support for primitive ("scalar") type hints reflection:

discover-ignored-tests