search

skinny85 / specnaz

Library for writing beautiful, RSpec/Jasmine/Mocha/Jest-style specifications in Java, Kotlin and Groovy
Other
34 stars 8 forks source link

When we get an unexpected exception Specnaz should print full message/stacktrace for it #12

Closed Stvad closed 4 years ago

Stvad commented 4 years ago

Right now only exception type is printed, which makes figuring out what actually happened rather hard

Related #2

skinny85 commented 4 years ago

Hey @Stvad ,

thanks for opening the issue!

So, the current situation is as follows. If we have a test like this:

public class JoinTest extends SpecnazJUnit {
    {
        describes("Join", it -> {
            it.shouldThrow(NoSuchElementException.class, "when calling joinStrings on null", () -> {
                Join.joinStrings(null);
            });
        });
    }
}

class Join {
    public static String joinStrings(List<String> strings) {
        return String.join(":", strings);
    }
}

the failure message is:

java.lang.AssertionError: Unexpected exception, expected: java.util.NoSuchElementException but was: java.lang.NullPointerException
Expected :java.util.NoSuchElementException 
Actual   :java.lang.NullPointerException
<Click to see difference>

    at org.specnaz.utils.ThrowableExpectations.verify(ThrowableExpectations.java:234)
    at org.specnaz.utils.ThrowableExpectations.access$000(ThrowableExpectations.java:50)
    at org.specnaz.utils.ThrowableExpectations$Wrapper.verify(ThrowableExpectations.java:260)
    at org.specnaz.impl.SingleExceptionTestCase.lambda$exercise$0(SingleExceptionTestCase.java:20)
    at org.specnaz.impl.SingleTestCase.invokeCallback(SingleTestCase.java:9)
    at org.specnaz.impl.SingleExceptionTestCase.exercise(SingleExceptionTestCase.java:19)
    at org.specnaz.impl.TestsGroupNodeExecutor.invokeTestBody(TestsGroupNodeExecutor.java:95)
    at org.specnaz.impl.TestsGroupNodeExecutor.runSingleTestCase(TestsGroupNodeExecutor.java:81)
    at org.specnaz.impl.TestsGroupNodeExecutor.runSingleTestCase(TestsGroupNodeExecutor.java:74)
    at org.specnaz.impl.TestsGroupNodeExecutor.lambda$null$0(TestsGroupNodeExecutor.java:33)
    at org.specnaz.impl.ExecutableTestCase.execute(ExecutableTestCase.java:21)
    at org.specnaz.junit.core.SpecnazCoreDslJUnitRunner$2.evaluate(SpecnazCoreDslJUnitRunner.java:226)
    at org.specnaz.junit.core.SpecnazCoreDslJUnitRunner.runTestsGroup(SpecnazCoreDslJUnitRunner.java:183)
    at org.specnaz.junit.core.SpecnazCoreDslJUnitRunner.run(SpecnazCoreDslJUnitRunner.java:167)
    at org.specnaz.junit.core.SpecnazCoreDslJUnitRunner.access$100(SpecnazCoreDslJUnitRunner.java:38)
    at org.specnaz.junit.core.SpecnazCoreDslJUnitRunner$1.evaluate(SpecnazCoreDslJUnitRunner.java:114)
    at org.specnaz.junit.core.SpecnazCoreDslJUnitRunner.run(SpecnazCoreDslJUnitRunner.java:121)
    at org.specnaz.junit.SpecnazJUnitRunner.run(SpecnazJUnitRunner.java:68)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
    at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

Clearly, the stack trace is from the AssertionError thrown because of the mismatch between the types of the expected exception and the received exception (code for it is here). There is no mention of the Join class in the stack trace.

The simplest solution is to add the received exception as the cause to the thrown AssertionError in the code above; that changes the error message to:

java.lang.AssertionError: Unexpected exception, expected: java.util.NoSuchElementException but was: java.lang.NullPointerException
Expected :java.util.NoSuchElementException 
Actual   :java.lang.NullPointerException
<Click to see difference>

    at org.specnaz.utils.ThrowableExpectations.verify(ThrowableExpectations.java:227)
    at org.specnaz.utils.ThrowableExpectations.access$000(ThrowableExpectations.java:50)
    at org.specnaz.utils.ThrowableExpectations$Wrapper.verify(ThrowableExpectations.java:255)
    at org.specnaz.impl.SingleExceptionTestCase.lambda$exercise$0(SingleExceptionTestCase.java:20)
    at org.specnaz.impl.SingleTestCase.invokeCallback(SingleTestCase.java:9)
    at org.specnaz.impl.SingleExceptionTestCase.exercise(SingleExceptionTestCase.java:19)
    at org.specnaz.impl.TestsGroupNodeExecutor.invokeTestBody(TestsGroupNodeExecutor.java:95)
    at org.specnaz.impl.TestsGroupNodeExecutor.runSingleTestCase(TestsGroupNodeExecutor.java:81)
    at org.specnaz.impl.TestsGroupNodeExecutor.runSingleTestCase(TestsGroupNodeExecutor.java:74)
    at org.specnaz.impl.TestsGroupNodeExecutor.lambda$null$0(TestsGroupNodeExecutor.java:33)
    at org.specnaz.impl.ExecutableTestCase.execute(ExecutableTestCase.java:21)
    at org.specnaz.junit.core.SpecnazCoreDslJUnitRunner$2.evaluate(SpecnazCoreDslJUnitRunner.java:226)
    at org.specnaz.junit.core.SpecnazCoreDslJUnitRunner.runTestsGroup(SpecnazCoreDslJUnitRunner.java:183)
    at org.specnaz.junit.core.SpecnazCoreDslJUnitRunner.run(SpecnazCoreDslJUnitRunner.java:167)
    at org.specnaz.junit.core.SpecnazCoreDslJUnitRunner.access$100(SpecnazCoreDslJUnitRunner.java:38)
    at org.specnaz.junit.core.SpecnazCoreDslJUnitRunner$1.evaluate(SpecnazCoreDslJUnitRunner.java:114)
    at org.specnaz.junit.core.SpecnazCoreDslJUnitRunner.run(SpecnazCoreDslJUnitRunner.java:121)
    at org.specnaz.junit.SpecnazJUnitRunner.run(SpecnazJUnitRunner.java:68)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
    at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.lang.NullPointerException
    at java.util.Objects.requireNonNull(Objects.java:203)
    at java.lang.String.join(String.java:2501)
    at Join.joinStrings(JoinTest.java:18)
    at JoinTest.lambda$null$0(JoinTest.java:10)
    at org.specnaz.impl.SingleTestCase.invokeCallback(SingleTestCase.java:9)
    at org.specnaz.impl.SingleTestCase.exercise(SingleTestCase.java:30)
    at org.specnaz.impl.SingleExceptionTestCase.exercise(SingleExceptionTestCase.java:18)
    ... 17 more

Now, the error message contains the stack trace of the received exception (with the Join class and all) in the Caused by: part.

Does this seem like a sensible solution to this problem @Stvad ?

Thanks, Adam

Stvad commented 4 years ago

Hey @skinny85 thanks for the prompt response! Yes that sounds reasonable :)

skinny85 commented 4 years ago

I've released this change as part of 1.5.2. I'll import it internally as soon as I can.

Stvad commented 4 years ago

Thank you, Adam!

skinny85 commented 4 years ago

My pleasure, thanks for opening the issue 🙂.

skinny85 commented 4 years ago

1.5.2 has been imported and released internally. I'm resolving this one, let me know @Stvad if you need anything else from my side for this issue.

Thanks, Adam