How to get stack traces on errors with raco test

[wiseman]

Given this program, testcase.rkt:

#lang racket

(module+ test
  (require rackunit))

(define (add)
  (+ 5 "4"))

(module+ test
  (test-case "add"
    (check-equal? (add) 9)))

When I do raco test testcase.rkt I get this output:

$ raco test testcase.rkt 
raco test: (submod "testcase.rkt" test)
--------------------
add
ERROR

+: contract violation
  expected: number?
  given: "4"
  argument position: 2nd
  other arguments...:
   5
--------------------
1/2 test failures

How can I get a stack trace showing me in which function (or even which line number) the error is happening?

Changing the require to this:

(module+ test
  (require errortrace rackunit))

Gives me this output:

$ raco test testcase.rkt 
raco test: (submod "testcase.rkt" test)
--------------------
add
+: contract violation
  expected: number?
  given: "4"
  argument position: 2nd
  other arguments...:
   5
  errortrace...:

+: contract violation
  expected: number?
  given: "4"
  argument position: 2nd
  other arguments...:
   5
--------------------
1/2 test failures

There's a new line, errortrace...: but it has no information.

(I suppose the output I've included does raise two other probably unrelated questions: 1. Why does it think there are 2 tests ("1/2 test failures")? 2. Why does requiring errortrace result in 6 duplicated lines of error message?)

Thanks!

[wiseman]

Can anyone else confirm that they either do or do not get stack traces for errors in rackunit tests?

[bennn]

My +1 was meant to confirm: "I see exactly what you're seeing with this, I thought about the problem for a minute, and I'm not sure what to do either"

[samth]

Just doing (require errortrace) is unlikely to be what you want. (See the documentation for errortrace for more info.) Instead, if you change #lang racket to #lang errortrace racket then you get a real stacktrace.

However, this still indicates several bugs, which I think are in Rackunit.

1. You still get 1/2 test failures.
2. You still get two exception printouts.
3. The macro expansion of check-equal? appears in the stack trace.

[wiseman]

Using #lang errortrace racket gives me a good stack trace, thank you! I always forget about #lang errortrace ...; I don't think it's actually documented in the errortrace docs. I don't know if you think it's worth mentioning in the rackunit documentation. I know it's not really a rackunit issue, but it would be nice if the "default" story for running unit tests (raco + rackunit) didn't require an additional, undocumented step to get usable stack traces.

[samth]

I'll add a line to the documentation for errortrace.

Note that you do get stack traces by default for raco test + rackunit -- the test case you have is very small and so they're empty. Here's an example:

[samth@huor:~/.../papers/pycket-papers/jfp-2016 (master) plt] raco test ~/e.rkt 
raco test: (submod "/home/samth/e.rkt" test)
error: fail
  context...:
   /home/samth/e.rkt:3:0: f
   (submod "/home/samth/e.rkt" test): [running body]
   for-loop
   run-module-instance!125
   /home/samth/sw/plt/pkgs/compiler-lib/compiler/commands/test.rkt:179:16
e.rkt: raco test: test raised an exception

#lang racket

(define (f x)
  (if (< (random 10) 2)
      (error 'fail)
      (add1 (f x))))

(module+ test
  (require rackunit)
  (check-equal? (f 100) 0))

[nano-o]

The example above does not work for me with Racket 8.3. What should I do to get error traces without #lang errortrace ...?

$ raco test test-rackunit-trace.rkt
raco test: (submod "test-rackunit-trace.rkt" test)
--------------------
ERROR
name:       check-equal?
location:   test-rackunit-trace.rkt:10:7

error: fail
--------------------
1/1 test failures

[williewillus]

In 8.8, raco test now has an --errortrace flag that should give a full backtrace.