Test reporting issue

[vincentjames501]

I haven't dug much into this, but I suspect this is related to how Cursive is dealing with clojure.test/report or clojure.test/try-expr.

This test works fine from a standard lein repl or via lein test but not via Cursive (reports 2 failures).

(defmacro ^:private wait-for
  "Waits for the given form to report no failures using 100ms intervals up to the specified time. 
  If it is still reporting failures after the specified time, report a failure."
  ([t form] `(wait-for ~t ~form nil))
  ([t form msg]
   `(loop [countdown# ~t]
      (let [events# (atom [])
            result# (binding [clojure.test/report
                              (fn [ev#] (swap! events# conj ev#))]
                      (clojure.test/try-expr ~msg ~form))]
        (if (and (seq (remove #(= (:type %) :pass) @events#))
                 (> countdown# 0))
          (do
            (Thread/sleep 100)
            (recur (- countdown# 100)))
          (do
            (doseq [ev# @events#]
              (clojure.test/report ev#))
            result#))))))

(deftest for-cursive
  (let [count (atom 0)]
    (testing "On our 3rd call to get-count, our test should pass"
      (letfn [(get-count [] (Thread/sleep 1000) (swap! count inc))]
        (wait-for 5000
                  (is (= 3 (get-count))))))))

Any suggestions on how I could get around this?

[cursive-ide]

I haven't had time to test this, but this is likely to be flaky under Cursive. I have to monkey-patch test/report to get the information I need to paint the test results in the editor - I suspect there's some interaction there. I'll try to reproduce this soon.

[vincentjames501]

No problem Colin. Keep up the awesome work!

[vincentjames501]

I suspect that a monkey patch on a monkey patch doesn't work so well!

[LukeInkster]

We have a fixture that binds clojure.test/report for capturing Selenium screenshots on test failure. When running outside Cursive this works fine but when running inside Cursive screenshots are not produced. Would it be possible for your monkey patch to capture and call back to report?

[vincentjames501]

+1

[cursive-ide]

Yes, I'm planning to try to make the Cursive test integration much less invasive - I'm hoping to get to this soon.

[dlebrero]

Any news on this?

[mk]

@cursive-ide we're using matcher-combinators heavily and would love to see it work better with cursive. According to a comment in slack this was being worked on last year. Is there anything we can do to help get this over the line?