Optionally don't print anything about skipped tests

[doublep]

Here is an implementation of the feature of "don't print anything about skipped tests" I proposed in issue #161. The diff might look somewhat big, but it is only addition of new code, there are no removals. This can be verified by rediffing it with whitespace ignored: all "remove" hunks are only because of reindenting.

Everything revolves around new variable buttercup-silent-skipping. I initialized it to nil to preserve current behavior, but personally think default value of t would be more logical.

When this variable is non-nil, all output from Buttercup is put on hold and kept in the list buttercup--pending-output. This is done because when we discover that a test is skipped, we need to "erase" already generated output, possibly going as far as not mentioning its suite at all (if everything in the suite is skipped). Other implementations are possible by not running anything skipped at all or changing order of events, introducing new events, etc., but this all would break backward compatibility. Proposed implementation only alters the built-in reporters, not touching the way tests are executed at all.

External reporters will not get skipping functionality for free, but they can implement something similar based on new function buttercup-spec-omitted-p.

I added two tests for new functions. Can write more if needed.

New functionality is available from command line too. Here is an example:

$ ./bin/buttercup -L . -p functions -s
Running 117 specs.

The `buttercup--apply-matcher' function
  should work with functions (34.13ms)

The Spy 
  `spy-on' function
    can spy on autoloaded functions (14.85ms)

Ran 2 out of 117 specs, 0 failed, in 0.1 seconds.
paul@gonzo:~/git/buttercup$ 

[doublep]

I will squash the new commit later, wanted to leave it visible for verification.

What do you think of #164 in comparison?

Looks fine for the problem in question. However, my approach would be extendable by altering buttercup-spec-omitted-p. You also have a function buttercup-omitted-p, but it is called before suite/spec is executed, restricting its options heavily.

What I specifically had in mind is adding another option and defvar buttercup-quiet. If used, only suites/specs that resulted in errors would be printed. This naturally can only be decided after running the tests and so your solution cannot be extended to support this, as far as I can see.

If idea of buttercup-quiet is rejected completely (even for future), then either solution is fine to me.

[doublep]

I've just had another idea: instead of putting output on hold, how about putting events on hold?

Outline: create a (defun buttercup-reporter-delaying-adapter (base-reporter) ...). It would return a reporter based on given one that would put events on hold and potentially discard them if not interesting according to "silent skipping" or "quiet" setting. If they become interesting (a test is not skipped / a test fails), event from the pending queue gets sent to the base-reporter.

Advantages:

• Over this (#162): easily reusable by other (external) reporters: just use (buttercup-reporter-delaying-adapter 'my-cool-reporter). But you still have to opt-in, so we don't break compatibility.

• Over #164: more open to extending, e.g. for buttercup-quiet.

Disadvantages:

• Reporters that do opt-in miss certain events completely as they get discarded. However, we can also convert them to sth. like SIGNAL-NAME:omitted so opting-in reporters still get them and may process them somehow should they want. This still wouldn't break compatibility, because such additional signals would only get sent to opting-in reporters, i.e. those aware of new functionality.

[doublep]

I'm closing this as #166 is certainly a better approach.