Midje test counts aren't shown

[philomates]

When you run tests via kaocha, unit tests are correctly counted in the summary at the end, but midje tests are not.

for instance, using the midje test runner on a project with both midje and clojure.test tests, you get the report:

>>> Midje summary:
All checks (219) succeeded.

>>> Output from clojure.test tests:

Ran 1 tests containing 6 assertions.
0 failures, 0 errors.

But using the kaocha runner on the same test suite results in

--- :unit ---------------------------
matcher-combinators.helpers-test

matcher-combinators.parser-test

matcher-combinators.printer-test

matcher-combinators.core-test

matcher-combinators.midje-test

matcher-combinators.matchers-test

matcher-combinators.test-test
  basic-matching

1 test vars, 6 assertions, 0 failures.

I stumbled upon https://github.com/plexus/kaocha-midje-test, and glancing at it, it looks like you are on the road to implementing this.

I didn't look to closely at your approach, but I wanted to offer that if there are any changes on midje that will make integrating with test runners such as kaocha easier, I will happily see if I can help in implementing them

Figured I'd open this so that midje users interested in using kaocha will know this is a current limitation.

[plexus]

Thanks a lot for trying this out! Currently out of the box only clojure.test style tests are supported (or anything that follows the same conventions of having :test metadata on a var.)

Midje support is next on the list though, the repo you pointed to contains my initial experiments for making this work. It seems to work, but I feel like I'll need to spend some time with Midje to better understand if I'm doing this right. (and maybe steal some feature ideas)

I'll have to try out if that code still works with the current Kaocha, but basically what you would do is

• put kaocha.type.midje on the classpath (just download it here)
• define a test suite with :type :kaocha.type/midje

#kaocha
{:tests [{:type :kaocha.type/midje
          :id :my-midje-test-suite
          :test-paths ["test"]}]}

That's it, the rest should work just like with clojure.test.

[plexus]

I updated the midje runner to reflect the latest kaocha, there was one breaking change in some configuration keys. It seems to work again, but tests are already run during the load change. (You can see that with bin/kaocha --print-test-plan, it shows test output before printing the test plan).

It also seems it's not compatible with the output capturing plugin, but that can be fixed.

If you could eyeball the code here especially in these two multimethods to see if anything looks weird that'd be super helpful.

(defmethod testable/-load :kaocha.type.midje/ns [{::keys [ns-name] :as testable}]
   )

(defmethod testable/-run :kaocha.type.midje/fact [{::keys [fact] :as testable} test-plan]
   )

[lambdaisland]

This was a matter of using the midje test type, which is now released, see https://github.com/lambdaisland/kaocha-midje

[adwelly]

I'm using:

:test {:extra-deps {lambdaisland/kaocha {:mvn/version "1.0.629"} lambdaisland/kaocha-midje {:mvn/version "0.0-5"}}}

with a test configuration of:

kaocha/v1 {}

{:tests [{:type :kaocha.type/midje :source-paths ["src"] :test-paths ["test"]}]}

(e.g. utterly vanilla - I hope). I'm seeing the problem of not counting the tests as well, although the tests are certainly being run. As far as I know this is the only way to run midje with deps.edn so I'm very grateful for that. I'm not importing midje directly in the deps.edn so presumably I'm transitively using whatever version kaocha-midje is picking up. Comments and advice appreciated.

[plexus]

Hi @adwelly, the Midje test type is really just a proof of concept, Midje is quite a complex thing to integrate with, so kaocha-midje is not really officially supported. My general recommendation is to migrate to Testit or Fudje. These both work on top of clojure.test and so you can use the default :kaocha.type/clojure.test.

If that's not an option for you then please create a minimal repo that reproduces your problem, describe expected and actual output, and I will make some time to have a look.

[adwelly]

I think either Testit or Fudge may be suitable, so I can probably work around; an opportunity to look at alternatives to Midje.

All the best

Andy

On 5 May 2020, at 11:03, Arne Brasseur notifications@github.com wrote:

Hi @adwelly https://github.com/adwelly, the Midje test type is really just a proof of concept, Midje is quite a complex thing to integrate with, so kaocha-midje is not really officially supported. My general recommendation is to migrate to Testit https://github.com/metosin/testit or Fudje https://github.com/jimpil/fudje. These both work on top of clojure.test and so you can use the default :kaocha.type/clojure.test.

If that's not an option for you then please create a minimal repo that reproduces your problem, describe expected and actual output, and I will make some time to have a look.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lambdaisland/kaocha/issues/4#issuecomment-623965940, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAT7ZBIOO7CQLQZ4ES6KIDTRP7QANANCNFSM4FVLON3Q.