Exceptions are printed out too verbosely

psexton commented 9 years ago

From @tgs on March 21, 2011 17:5

The stack trace doesn't have anything to do with the actual test failure:

===== Test Case Failure =====
Location: /home/tgs/multi-lib/data-analysis/cevent/cevent_relative_intervals.m
Name:     cevent_relative_intervals

/home/tgs/src/xunit-git/xunit/DocTest.m at line 203
/home/tgs/src/xunit-git/xunit/DocTest.m at line 100
/home/tgs/src/xunit-git/xunit/TestSuite.m at line 85
/home/tgs/src/xunit-git/xunit/TestSuite.m at line 85
/home/tgs/src/xunit-git/xunit/TestSuite.m at line 85
/home/tgs/src/xunit-git/xunit/runtests.m at line 126

DocTest #4 from cevent_relative_intervals failed; expected first, got second
Inputs are not equal.

First input:
ans = 1 10 1 21 21 2 81 85 3

Second input:
??? Error using ==> cevent_relative_intervals at 76 The WHENCE argument must be either 'start' or 'end'.

Copied from original issue: psexton/matlab-xunit#1

psexton commented 9 years ago

From @tgs on March 21, 2011 17:27

Grr. The stack field of MException is read-only. Hard to work around that.

psexton commented 9 years ago

From @tgs on March 21, 2011 17:32

Might be able to edit filterStack in TestRunDisplay.m to fix this.

psexton commented 9 years ago

From @tgs on April 22, 2011 20:29

OK, so:

The stack field of MException is very thoroughly read-only, not even child classes can change it.
DocTests have a different sort of stack than regular unit tests

So I think the best option is to change how the stack is printed depending on whether the test is a doctest or a unit test.

What I want:

If the doctest actually threw an unexpected exception, print the stack trace from that. But that should probably be filtered so the DocTest runner isn't in it.
If it was a test failure because the expectation wasn't met, don't print the stack trace, just print the expectation and result, and give a link to the relevant place in the function with the doctest.

psexton commented 9 years ago

From @tgs on April 22, 2011 20:34

Yet another option would be to:

filter DocTest stuff from the exception stack
make the message string in the assertEqual at xunit/DocTestCase.m:202 be super long and descriptive, with a link and everything

psexton / matlab-xunit-doctests

Exceptions are printed out too verbosely #1