Feature request: Show what exact assertion has failed

[underdpt]

Hi

I have some long tests that covers full user interaction with some functionality. There may be tens of assertions on a single test, and I'm facing some difficulty getting to know which assertion has failed.

Take this example:

test('multiple assertions', function() {

    /** do something */
    assertTrue(true);

    /** do something more... */
    assertTrue(false);

    /** finish things up... */
    assertTrue(false);

})->only();

Upon running the test, this is the output:

> .\vendor\bin\pest

   FAIL  Tests\Feature\ExampleTest
  ⨯ multiple assertions

  ---

  • Tests\Feature\ExampleTest > multiple assertions
  Failed asserting that false is true.

  at D:\...\...\...\web\vendor\phpunit\phpunit\src\Framework\Constraint\Constraint.php:119
    115▕         if (!empty($description)) {
    116▕             $failureDescription = $description . "\n" . $failureDescription;
    117▕         }
    118▕
  ➜ 119▕         throw new ExpectationFailedException(
    120▕             $failureDescription,
    121▕             $comparisonFailure
    122▕         );
    123▕     }

  1   D:\...\...\...\web\vendor\phpunit\phpunit\src\Framework\Constraint\Constraint.php:53
      PHPUnit\Framework\Constraint\Constraint::fail("")

  2   D:\...\...\...\web\vendor\phpunit\phpunit\src\Framework\Assert.php:2228
      PHPUnit\Framework\Constraint\Constraint::evaluate("")

  Tests:  1 failed
  Time:   9.36s

It's impossible to know which assertion has failed, I have to add more verbosity and then:

> .\vendor\bin\pest -v

   FAIL  Tests\Feature\ExampleTest
  ⨯ multiple assertions

  ---

  • Tests\Feature\ExampleTest > multiple assertions
  Failed asserting that false is true.

  at D:\...\...\...\web\vendor\phpunit\phpunit\src\Framework\Constraint\Constraint.php:119
    115▕         if (!empty($description)) {
    116▕             $failureDescription = $description . "\n" . $failureDescription;
    117▕         }
    118▕
  ➜ 119▕         throw new ExpectationFailedException(
    120▕             $failureDescription,
    121▕             $comparisonFailure
    122▕         );
    123▕     }

  1   D:\...\...\...\web\vendor\phpunit\phpunit\src\Framework\Constraint\Constraint.php:53
      PHPUnit\Framework\Constraint\Constraint::fail("")

  2   D:\...\...\...\web\vendor\phpunit\phpunit\src\Framework\Assert.php:2228
      PHPUnit\Framework\Constraint\Constraint::evaluate("")

  3   D:\...\...\...\web\vendor\phpunit\phpunit\src\Framework\Assert.php:1022
      PHPUnit\Framework\Assert::assertThat(Object(PHPUnit\Framework\Constraint\IsTrue), "")

  4   D:\...\...\...\web\vendor\pestphp\pest\compiled\globals.php:860
      PHPUnit\Framework\Assert::assertTrue()

  5  D:\...\...\...\web\tests\Feature\ExampleTest.php:8
      assertTrue()

  6   D:\...\...\...\web\vendor\pestphp\pest\src\Factories\TestCaseFactory.php:142
      P\Tests\Feature\ExampleTest::{closure}()
....deleted....

I can see on line 5 that the assertion that has failed is on line 8.

Compared with phpunit:

class SecondTest extends TestCase
{
    public function testBasicTest()
    {
        /** do something */
        $this->assertTrue(true);

        /** do something more... */
        $this->assertTrue(false);

        /** finish things up... */
        $this->assertTrue(false);
    }
}

Results:

> .\vendor\bin\phpunit --filter=SecondTest
PHPUnit 8.5.8 by Sebastian Bergmann and contributors.

F                                                                   1 / 1 (100%)

Time: 181 ms, Memory: 8.00 MB

There was 1 failure:

1) Tests\Unit\SecondTest::testBasicTest
Failed asserting that false is true.

D:\...\...\...\tests\Unit\ExampleTest.php:20

FAILURES!
Tests: 1, Assertions: 2, Failures: 1.

It's pretty quick to know which assertion failed.

I think it could be really useful to state the line with the failed assertion on the output.

[nunomaduro]

That must be a bug here: https://github.com/nunomaduro/collision/blob/1e43bcab2ce79146b9852ca81a25ae2d1d21986b/src/Writer.php#L204.

Can you try to fix it?

[olivernybroe]

@underdpt Is this fixed now with the pr to collision?

[underdpt]

Yes, I'm sorry I forgot to close this issue.

[olivernybroe]

No problem, just double checking 👍