search

kotest / kotest-gradle-plugin

A gradle plugin for Kotest
https://kotest.io
Apache License 2.0
29 stars 5 forks source link

Unable to see failures summarized at the end #11

Open rezammalik opened 3 years ago

rezammalik commented 3 years ago

I am not seeing failures summarized at the end of the test runs. I am using this with Gradle 4.4.3 though. Is this something that would work only with Gradle version specified in the README?

sksamuel commented 3 years ago

Gradle 4.4.3 is quite old. This plugin is only targeting 6.x Can you upgrade? If you're using the gradle wrapper it should be straightforward?

rezammalik commented 3 years ago

oh, I misspoke. I meant I am using Kotest 4.4.3 and Gradle 6.8.2.

sksamuel commented 3 years ago

So like a section that sums it up ? The reporter should already be doing that. Do you have a screenshot ?

rezammalik commented 3 years ago

Here is a use case, 4 tests, 2 per Class are run.

The first one is good with both tests passing, the second one has 2 assertions in 1 test fail (reported together via softAsserts), the second test in the file just has one assert fail, shows correctly. 

2.  testy.TestClass2
    + Account created via endpoint
        + for user2
            + on Internal Endpoint
                - returns the correct response and passes data validation OK (1411ms)
            + on Gateway Endpoint
                - returns the correct response and passes data validation OK (1577ms)

3.  testy.TestClass1
    + Account created via endpoint
        + for user 1
            + on Internal Endpoint
                - returns the correct response and passes data validation FAILED (1303ms)

The following 2 assertions failed:
1) expected:<"FAILED"> but was:<"SUCCESS">
        at testy.TestClass1$1$1$2$1$1.invokeSuspend(TestClass1.kt:82)
2) expected:<"FAILED"> but was:<"SUCCESS">
        at testy.TestClass1$1$1$2$1$1.invokeSuspend(TestClass1.kt:84)

<... stack trace here>

            + on Gateway Endpoint
                - returns the correct response and passes data validation FAILED (1579ms)

expected:<"FAILED"> but was:<"SUCCESS">
<... stack trace here>

>> There were test failures

 testy.TestClass1
 - Account created via endpoint for user 1 on Internal Endpoint returns the correct response and passes data validation
 - Account created via endpoint for user 1 on Gateway Endpoint returns the correct response and passes data validation

Specs:   2 passed, 1 failed, 3 total
Tests:   2 passed, 2 failed, 0 ignored, 4 total
Time:    27s

> Task :kotest FAILED

FAILURE: Build failed with an exception.

I was expecting the the summary at the end to list the assertions that failed, like in the readme.md screengrabs. Just wondering if my expectations are off or there is something I am not doing correctly, for them to now show up.

sksamuel commented 3 years ago

I get you.

Good idea I'll add it.

On Thu, 25 Mar 2021, 16:05 Reza Malik, @.***> wrote:

Here is a use case, 4 tests, 2 per Class are run.

The first one is good with both tests passing, the second one has 3 assertions in 1 test fail (reported together via softAsserts), the second test in the file just has one assert fail, shows correctly.

  1. testy.TestClass2

    • Account created via endpoint
      • for user2
        • on Internal Endpoint
          • returns the correct response and passes data validation OK (1411ms)
        • on Gateway Endpoint
          • returns the correct response and passes data validation OK (1577ms)

  2. testy.TestClass1

    • Account created via endpoint
      • for user 1
        • on Internal Endpoint
          • returns the correct response and passes data validation FAILED (1303ms)

The following 2 assertions failed: 1) expected:<"FAILED"> but was:<"SUCCESS"> at testy.TestClass1$1$1$2$1$1.invokeSuspend(TestClass1.kt:82) 2) expected:<"FAILED"> but was:<"SUCCESS"> at testy.TestClass1$1$1$2$1$1.invokeSuspend(TestClass1.kt:84)

<... stack trace here>

        + on Gateway Endpoint
            - returns the correct response and passes data validation FAILED (1579ms)

expected:<"FAILED"> but was:<"SUCCESS"> <... stack trace here>

There were test failures

testy.TestClass1

  • Account created via endpoint for user 1 on Internal Endpoint returns the correct response and passes data validation
  • Account created via endpoint for user 1 on Gateway Endpoint returns the correct response and passes data validation

Specs: 2 passed, 1 failed, 3 total Tests: 2 passed, 2 failed, 0 ignored, 4 total Time: 27s

Task :kotest FAILED

FAILURE: Build failed with an exception.

I was expecting the the summary at the end to list the assertions that failed, like in the readme.md screengrabs. Just wondering if my expectations are off or there is something I am not doing correctly, for them to now show up.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/kotest/kotest-gradle-plugin/issues/11#issuecomment-807485584, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFVSGWO5CPTQRQCROQ53J3TFOQRBANCNFSM4ZBBWYLA .

rezammalik commented 2 years ago

@sksamuel Hey Sam. Is this something that is available now or will be soon?