Function spec checking should be opt-in

[mmmdreg]

Since 0.0-266, kaocha is using orchestra for checking function specs.

In my codebase, I use fixtures to enable spec instrumentation per namespace as needed. This is because in some instances I do not care if my specs are strictly valid (e.g using some dummy data to test methods against a DB for filtering/sorting etc where a field might be “A” instead of some string matching some stricter regex as defined in a spec).

Because I opt in via my code, running tests in cursive or via lein test are equivalent.

Kaocha by default adds spec instrumentation across my entire codebase, which is intrusive and also means it is not a drop in test runner replacement.

I would like this to be disabled by default or at least able to be turned off. Ultimately kaocha shouldn’t force me to rewrite code just to be able to use it.

[kennyjwilli]

I have been looking at replacing our test runner with kaocha. This issue would definitely block us from using it.

[plexus]

A PR to make this configurable in tests.edn would be welcomed!

On Wed, Jul 17, 2019, 18:57 Kenny Williams notifications@github.com wrote:

I have been looking at replacing our test runner with kaocha. This issue would definitely block us from using it.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/lambdaisland/kaocha/issues/104?email_source=notifications&email_token=AAAH3VF3L4INMYO6S7I4TJ3P75FOHA5CNFSM4H6XSSO2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2FB22Q#issuecomment-512367978, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAH3VADIBLVGABSNQVEXJLP75FOHANCNFSM4H6XSSOQ .

[AndreaCrotti]

Isn't it what :kaocha.spec.test.check/instrument? is for? It's enabled by default (which I think it's a good thing), but you can disable it right? Or I misunderstood?

[plexus]

That option is only relevant if you're using the stc plugin.

[WhittlesJr]

I can look into this in the coming weeks if no one else is planning on doing so

[danielcompton]

It seems like constraining the instrumentation to just Kaocha's namespaces could be a good option here to avoid causing problems for application test code.

This would get you a collection of Kaocha symbols:

(->> (orchestra.spec.test/instrumentable-syms)
     (filter #(str/starts-with? (namespace %) "kaocha.")))

Not sure if there is a better/cleaner method than this though?

[plexus]

Yes, that would definitely be a good thing to do

[humorless]

I tried to understand this issue, but what I found in kaocha's codebase is that orchestra is already a plugin. Also, the changelog said so.