racket / redex

Other
93 stars 36 forks source link

where/error failures give no information about source of error #250

Closed nikomatsakis closed 3 years ago

nikomatsakis commented 3 years ago

I had the following code:

   (where/error Store_0 (apply-action Store Action_0))

The correct code was:

   (where/error Store_0 (apply-action-to-store Store Action_0))

The error I got when running my tests was:

> raco test racket/z-tests/opsem-action.rkt 
raco test: "racket/z-tests/opsem-action.rkt"
define-metafunction: where/error did not match
  context...:
   /usr/share/racket/pkgs/redex-lib/redex/private/judgment-form.rkt:419:0: combine-where/error-results
   /usr/share/racket/collects/racket/private/map.rkt:35:13: map
   /usr/share/racket/pkgs/redex-lib/redex/private/reduction-semantics.rkt:1868:24: loop
   /usr/share/racket/pkgs/redex-lib/redex/private/reduction-semantics.rkt:86:30
   "/home/nmatsakis/versioned/dada-model/racket/z-tests/opsem-action.rkt": [running body]
   /usr/share/racket/pkgs/compiler-lib/compiler/commands/test.rkt:179:16
opsem-action.rkt: raco test: test raised an exception
3 tests passed

The error doesn't identify any of my code, but it also doesn't include context like the pattern or value that was being matched. Anything along these lines would be helpful! Knowing the filename/line-number where the where/error appears would be ideal.

I have observed that the vast majority of where/error failures are of this kind: either I have a typo in the name of a function, or else I have a failure to import the module that contains the function. The remainder are probably outdated patterns where I've tweaked something in the signature. It would be nice to target diagnostics for at least this relatively simple case!

rfindler commented 3 years ago

Hi Niko: have you tried running with errortrace enabled (perhaps in DrRacket)?

rfindler commented 3 years ago

I got an offline messages from someone else suggesting I should elaborate a little bit. It may be that you're running outside of DrRacket / with errortrace turned off for performance reasons. And redex code seems to benefit more than other code for doing that so that makes sense as a default. Redex, however, has a bunch of code to cooperate with errortrace so that the source locations are helpful when errortrace is turned on. So my recommendation would be to temporarily turn it back on to get a source location for the error when things like this happen. If running in DrRacket isn't convenient for some reason, then a command line invocation like this one:

racket -l errortrace -l racket/base -e '(require (submod "x.rkt" test))'

where "x.rkt" is replaced with your filename should also do the trick.

(If there is a situation where using errortrace like that still gives a bad source location, that's a bug (and fixable), so please send me something to reproduce it so I can find it (it is okay if it is a lot of code; that won't matter for this kind of bug, I believe.)

nikomatsakis commented 3 years ago

@rfindler I'm currently using raco outside of dr-racket because it's running on a server without X11 enabled. I can certainly try enabling errortrace; is the -l option something I can also pass to raco?

rfindler commented 3 years ago

You'd need to use a command-line like the racket one I posted, I believe. raco test runs test submodules, and so that's why there is the require there. There may be a more convenient way to do this, but I'm not sure what it is. I don't see an argument to raco test that turns errortrace on (but adding one does seem like a good idea).

If you're running in CI or something like that, you might want to leave errortrace out of it and use errortrace only for debugging locally on your machine, tho.

nikomatsakis commented 3 years ago

OK, next time I get an error like this, I will give it a try, thanks.

I am wondering: how hard would it be for the where/error failure message to include more information? That would already be enough a lot of the time.

nikomatsakis commented 3 years ago

OK, I tried it, and it worked! Thanks.