Add ability to customize test output when a test fails

[shepmaster]

I would like the ability to control / enhance / replace the output generated when a test fails. Printing the raw values works well as long as those values have a useful output for the test at hand, but this doesn't always hold true.

To give a bit of background, I was playing with generating sequences of partially-applied functions and then applying them to an initial state. Here's a small excerpt from the full gist:

:smallest [(#<core$partial$fn__4190 clojure.core$partial$fn__4190@2a799171>)]

What would be more useful to me is if the output had something like this, which is more meaningful:

:smallest [(- 45 %)]

In this example, (- 45 %) is all my custom output string. 45 is a value from an upstream generator (and would thus change), and % is just a string modeled after the anonymous function syntax.

I could even see my particular application rolled up as a generator:

(def gen-inc
  (gen/bind gen/pos-int #(gen/partial + %)))

Which could handle some of the heavy lifting for the user.

[shepmaster]

Continuing to work on my problem, I created a hacky solution. I think that two useful potential solutions came of it:

1. Create a parallel implementation of print-method for testing purposes. I don't know enough about pprint, but maybe that would also be something to hook into?
2. Annotating the generated values with metadata.

In the linked solution above, I do a combination of both. I override the existing print-method for functions and then check for my own generated metadata to display instead.

[cemerick]

I like the notion of using metadata to help provide information about unreadable generated values. But, I think making things print prettily should be a non-goal. While it might be aesthetically pleasing to see (- 45 %), it's probably better in every other way for the metadata to be something other than a string for printing, e.g. {:partial-fn '- :args [45]} in your case.

[shepmaster]

I'm not opposed to your suggestion, but I think there's middle ground. For example, simple-check could have a multimethod describe function that takes the value and returns another value.

The default implementation would return some metadata key if present or just identity otherwise. This should preserve the existing behavior, if I understand correctly. The metadata would be '(partial - 45) for my case.

I agree that focusing on a string is a poor choice, but perhaps you wouldn't mind sharing a bit more on why your breakdown is yet a better way?

In this case, I'm still focusing on making the output very easy for a human to parse and understand. I think this falls under aesthetics.

[reiddraper]

Moved to Jira.