Actual error display in html reports

[GoogleCodeExporter]

It would be very useful if the html reports could show on demand the actual 
error that made a scenario fail. E.g. the error text could be shown by clicking 
the red "Failed" icon, as Hudson/Jenkins do for the failing tests.

So far the only way to see what went wrong is by reading the logs and the files 
generated in the target/surfire-reports directory.

Original issue reported on code.google.com by al...@fastmail.net on 19 May 2011 at 4:46

[GoogleCodeExporter]

Valid comment. Will think how to best solve it.

Original comment by lipinski...@gmail.com on 27 May 2011 at 7:21

• Changed state: Accepted

[GoogleCodeExporter]

I came across a similar need, and I use JUnit MethodRule as a work around

something like this as a MethodRule implemtention

{{{

package tumbler.experimental;

import org.junit.rules.MethodRule;
import org.junit.runners.model.FrameworkMethod;
import org.junit.runners.model.Statement;

public class BddRule implements MethodRule {

  @Override
  public Statement apply(final Statement base, FrameworkMethod method, Object target) {
    return new Statement() {
      public void evaluate() throws Throwable {
        try {
          base.evaluate();
        } catch (Throwable t) {
          unexpectedThen(t.getMessage());
          throw t;
        }
      }
    };
  }

  private void unexpectedThen(String message) {
    tumbler.Tumbler.Then(message);
  };

}

}}}

then my scenario looks like this (here, I'm testing a 'bugged' add function 
which substracts instead of adding) 

{{{

package tumbler.experimental.sample;

import static tumbler.Tumbler.Given;
import static tumbler.Tumbler.Then;
import static tumbler.Tumbler.When;
import junit.framework.Assert;

import org.junit.Rule;
import org.junit.rules.MethodRule;
import org.junit.runner.RunWith;

import tumbler.Scenario;
import tumbler.Story;
import tumbler.TumblerRunner;
import tumbler.experimental.BddRule;

@RunWith(TumblerRunner.class)
@Story("Basic Operations")
public class SomeScenario {

  @Rule
  public MethodRule tumblerFailureRule = new BddRule();

  @Scenario("Addition validation")
  public void shouldDoSomething() {
    int a = 1;
    int b = 2;
    Given("operands a=" + a + " and b=" + b);

    When("I add a and b");
    int actual = add(a, b);

    int expected = 3;
    Assert.assertEquals(expected, actual);
    Then("expected result is=" + expected);
  }

  static int add(int a, int b) {

    // system under test is bugged : substract instead of add
    return a - b;
  }
}

}}}

as a result, i can see directly, in the *html report*, at the *Then* place, the 
Exception message raised by Junit *assertEquals*, namely _expected:<3> but 
was:<-1>_  

Original comment by pascal.b...@gmail.com on 1 Jun 2011 at 3:23

[GoogleCodeExporter]

Ok, this was a nice way of getting that exception, now you have a normal one ;-)
Exception object is stored in the scenario status. So in your report template 
you can take it by calling scenario.status().details()

Currently I've set it as icon (stop sign) title (when mouse is over it).

Original comment by lipinski...@gmail.com on 13 Jun 2011 at 4:08

• Changed state: Fixed

[GoogleCodeExporter]

Great, thanks !

Original comment by pascal.b...@gmail.com on 14 Jun 2011 at 8:06

[GoogleCodeExporter]

Great! Is that showing the full stack trace?

Shall we be able to see it in the upcoming 0.3.1 release?

Original comment by al...@fastmail.net on 15 Jun 2011 at 6:02

[GoogleCodeExporter]

This is exactly the Exception object. Scenario.status().details() returns 
Exception object for failed scenarios.
It's already in 0.3.1-SNAPSHOT in sonatype's snapshot repo. I'll release a 
full-numbered version still this week with all the stuff inside.

Original comment by lipinski...@gmail.com on 15 Jun 2011 at 6:08

[GoogleCodeExporter]

It might not be enough. The exception and the stack trace say what has 
happened, but not where.

When JUnit tests fail, it's necessary to have a look at JUnit output to see 
where the failure occurred. Not all failures are due to exceptions. They are 
mostly due to failing assertions.

Is this going to show that as well?

Original comment by al...@fastmail.net on 16 Jun 2011 at 7:43

[GoogleCodeExporter]

When an assertion fails, you get AssertionException. Everything is in the 
exception.

Original comment by lipinski...@gmail.com on 16 Jun 2011 at 8:42