Could we store the test result in the results that are passed to the post-summary hook?

[gabrielgiussi]

Right now, at least with version v1, the result of each test received by the post-summary hook contains the following keys

(:kaocha.result/error
 :kaocha.result/pending
 :kaocha.result/pass
 :kaocha.var/test
 :kaocha.plugin.randomize/sort-key
 :kaocha.testable/events
 :kaocha.plugin.capture-output/output
 :kaocha.testable/meta
 :kaocha.result/count
 :kaocha.testable/type
 :kaocha.var/name
 :kaocha.var/var
 :kaocha.testable/desc
 :kaocha.testable/wrap
 :kaocha.result/fail
 :kaocha.testable/id)

I would like to have access to the test result, assuming kaocha is calling the function under :test we would be storing true for this simple case

(deftest simple-test
  (is (= 1 1)))

((:test (meta #'simple-test)))
; => true

However, if you run a test using the defflow macro from state-flow the result is a pair [result state] which would allow me to store information in the state that I can later use to build a report in the post-summary hook.

(defflow my-flow {:init (constantly {:value 1
                                     :map   {:a 1 :b 2}})}
  [value (state/gets :value)]
  (testing "1" (mc/match? 1 value))
  (testing "b is 2" (mc/match? {:b 2} (state/gets :map))))

((:test (meta #'my-flow)))
; => [{:match/result :match,
;  :match/expected {:b 2},
;  :match/actual {:a 1, :b 2},
;  :probe/results [{:check-result true, :value {:a 1, :b 2}}],
;  :probe/sleep-time 200,
;  :probe/times-to-try 1}
; {:value 1, :map {:a 1, :b 2}}]

[plexus]

Hi Gabriel,

A deftest is normally only ever called for side effects, the fact that it returns its last value is an implementation detail that is ignored by test runners.

The only way you would be able to access that return value is by adding a :kaocha.testable/wrap key to the var testable, from a hook or plugin.

[gabrielgiussi]

Hi Arne, I'm testing your proposal but it is not entirely clear for me how to assoc the test result to the map we receive later in the post-summary. I don't see any example for the :kaocha.testable/wrap hook but I'm testing with this

(defn wrapped [test]
  (fn []
    (let [result (test)]
      result)))

(defn wrap [test test-plan]
  (println "test" (type test))
  (assoc test :kaocha.testable/wrap [wrapped]))

In the wrapped function I have access to the test result, but the only thing I can do with it at that point is to return it. I don't have a way to assoc it under a key to the result because is not that we are returning the map with keys :kaocha.result/error, kaocha.result/pending, :kaocha.result/pass, etc.

[gabrielgiussi]

I guess the proposal was to store the result in an atom and then read it in the post-summary hook.