Integrate with existing test frameworks

[DalekBaldwin]

There are a bunch of test frameworks out there with their own peculiar interfaces for declaring e.g. that a test case is expected to signal a particular error. There's no reason to attempt a half-assed duplication of those kinds of features with their own idiosyncratic API that people have to learn just to add a few Quickcheck-style tests to their suite using similar features. I'd like the usage pattern to stay as it is, so you can just include a check-it form within a test created with whatever framework you already know and like.

However, generating code for a deterministic test with :gen-output-template and writing it to a file is kinda sketchy. It requires you to redundantly write a testing-framework-level test form within a testing-framework-level test form, and it could probably break in weird ways under unforeseen circumstances. It would be better just to print a readable representation of the data that caused a test to fail and tag it with an ID corresponding to the failed test. But that would require the check-it macro to detect what sort of context it's being used in, so it could associate the name of the test with the correct piece of serialized data to replicate an old test case. It would need to know something about the top-level test definition form surrounding it, or else it would need an extra symbol as a parameter to tag its failure cases.

I think the simplest way to combine test framework integration and sane failure data serialization is to create specific wrappers for popular test frameworks' existing test-defining macros. So if you wanted to make use of a check-it form, you would define the test with deftest-with-check-it instead of deftest. These wrappers would encode the name of the test in a place the check-it macro knows where to look for. The check-it macro would then expand into a form that looks up the existing failure cases, runs them, and then runs new randomized cases.

The failure output file and package could then be declared in a single location, possibly in another wrapper macro surrounding defsuite (or the corresponding macro in whatever test framework).

[DalekBaldwin]

For now I'm just doing basic data serialization and accepting the cost of a little repetition on the user's end. This should keep things simple enough that people may actually want to try it out and give feedback about how best to integrate it with frameworks.

[DalekBaldwin]

Most importantly, check-it will need to properly handle expected/unexpected errors and be able to propagate those expectations downward for effective shrinkage. I don't know if that can be integrated with a testing framework's built-in error expectation protocols.