lein midje configuration: non-zero exit code for "No facts were checked."

[bcmreed]

I would like to be able to force the "No facts were checked." condition to return a non 0 exit code.

In my opinion, the default exit code for this condition should not be 0, but I'd be perfectly happy with a configuration option to force the exit code.

Perhaps instead of a boolean "force-no-facts-error" config option, you could go with a "min-facts" option, in which lein midje will fail if less facts than that value are checked.

sample:

$ lein midje
...
No facts were checked. Is that what you wanted?

$ echo $?
0

Additional context:

We use midje for automated tests run in teamcity. We had a one-off test that always looked like it passed. It wasn't until I was debugging something else that I realized that the result of the test was actually not a success, but "No facts were checked." (and the 0 exit code kept everything looking "green"). I had accidentally renamed some filters, and forgot about the command that this test used was using a now defunct filter.

I realize there are many other things I could have done to prevent this from happening, but it would be nice to force the exit code and failure.

[marick]

Let me see if I understand you. You'd like:

1) the output of the load-facts function to return a :failure count > 0 if any of the loaded namespaces has no facts.

2) ... with the result that lein midje will have a non-zero exit status in such a case.

Right?

[bcmreed]

I am more directly concerned about when no facts are run at all (as opposed to no facts in any of the loaded namespaces). But if #1 is the logical step to accomplish #2, then that sounds good to me.

In other words: if I run load-facts or lein midje, my intention if for tests to be run. If no tests are run, I get an info message "No facts were checked. Is that what you wanted?" and no error exit code. I'd like to force the answer to the question "Is that what you wanted?" to be "no!".

[marick]

Linked to trello board https://trello.com/c/T439rdCS/9-lein-midje-configuration-non-zero-exit-code-for-no-facts-were-checked-253