weavejester / eftest

Fast and pretty Clojure test runner
424 stars 40 forks source link

Excessive printing by `eftest.report.progress/report` on `:pass` #94

Open p-himik opened 2 months ago

p-himik commented 2 months ago

Just spent half a day chasing an intermittent issue where running a bunch of quick tests with 30k assertions would sometimes result in the behavior on the recording below. It would last for at least multiple minutes, whereas a normal run would take 20 seconds.

https://github.com/user-attachments/assets/3368f429-2784-4021-a032-ab8b87d92931

After experimenting with multiple changes to the code, debugging things, changing reporters, changing terminals and shells used to run the tests, I'm now 90% certain that the culprit is some rendering-related system on my end. Maybe it's the GPU driver, maybe it's the hardware, maybe X11, maybe something else.

But the issue goes away if I make this small addition to my code:

(let [orig-report (get-method eftest.report.progress/report :pass)]
  (defmethod eftest.report.progress/report :pass [m]
    (binding [clojure.test/*test-out* (java.io.StringWriter.)]
      (orig-report m))))

As a bonus, the tests now run 1-2 s faster. Visually, there are no changes at all since the report of :pass doesn't add any new information anyway - all the information is already printed by :begin-test-run and :end-test-var.