search

gfredericks / test.chuck

A utility library for test.check
Eclipse Public License 1.0
214 stars 26 forks source link

clojure-test: `::shrunk` report type is not very portable #66

Open frenchy64 opened 3 years ago

frenchy64 commented 3 years ago

It turns out clojure.test/report is very flexible, being both a dynamic var and a multimethod. I think introducing a custom reporting event via :type ::shrunk is pretty much impossible to be compatible with this dynamism.

For example, eftest has 3 different multimethods, all with different output formats: ANSI colors, xml, and progress bar. It dynamically rebinds clojure.test/report based on configuration parameters.

What this means is the ::shrunk printing disappears when the reporting mechanism is rebound. And since test.chuck can't predict what the current output format is, it seems wiser to piggieback the smallest+seed reporting on existing reporting types like :fail or :pass.

gfredericks commented 3 years ago

If there's a refactoring that roughly preserves the current vanilla user experience, I'd be open to it.

frenchy64 commented 3 years ago

Hmm, one way to achieve that might be if (thread-bound? #'ct/report), then propagate the reporting to individual test reports, otherwise keep the current format. I think the logic could be held in c.g.t.chuck.clojure-test/-report.