Testing Plan for Type Errors & (Basic) Type Errors Test Suite

[RiugaBachi]

As discussed in this comment of #120:

@RiugaBachi If you could add a test plan document (it could even live in ./docs/...) that will instruct me what to do (the changes to make to current examples, for instance) to trigger all possible custom type errors in this PR, that would be great. I can use that to review this PR, as well as to test future releases.

This PR adds the proposed testing plan document residing under docs/testing.md, as well as a test-type-errors test suite that currently houses an (informal, manually-verified-for-now) specification for custom errors related to route generic deriving. This has been designed to be as painless as possible to automate in the future via some minimal TH plumbing, while still being plenty legible for human eye verification.

These tests have caught a few unhandled cases / regressions in #120 as well, which I plan to submit a PR for separately after this is resolved. If the expected error does not line up what what GHC outputs, this is probably the reason why.

[RiugaBachi]

Some personal thoughts on the current usage of undefined:

• It raises eyebrows for sure.
• For manual verification, it serves well enough; i.e., there's technically no issue with it. Afaik.
• I'm not 100% sure how this will play in the future when automating, but defining trivial implementations for IsRoute should be fairly easy.
• How I foresee it playing out:
  • For test cases looking for an error, undefined will never be evaluated as the stack unwind will be triggered by the function not typechecking to begin with.
  • For test cases that verify that there is no compiler error, there is no need to really evaluate these at runtime. The fact that the test suite compiles at all is verification enough that it passes this kind of spec. Hence, we won't evaluate undefined in this case either.

[RiugaBachi]

• I feel like we should define the routes manually (by hand), just as the model types are being defined. niceRoute and badRoute seem like abstractions we can do away with.

This is true, I didn't end up using as many custom routes as I thought I was going to.

• I'm unclear on if we need to test the Type rep field name lookup and Type rep indexing cases. How often do they happen in a real-world scenario?

Probably not a lot and frankly I was being pedantic. Once this becomes automated though, it shouldn't be a concern, so for now would you saying labeling these as low priority for manual testing would be good?

• Instead of having to uncomment every snippet by hand, can we use CPP to toggle them on and off with a single line change? I plan to rely on HLS (rather than repl) to test it, and I imagine I could just use the hover tooltip to read the error.

Good idea actually.

[RiugaBachi]

An update on the original issues:

I feel like we should define the routes manually (by hand), just as the model types are being defined. niceRoute and badRoute seem like abstractions we can do away with.

I realized the reason I did this in the first place was because we're testing the same data structure but with different deriving clauses, so if we do write it out by hand that's n route types + naming conflicts to deal with.

I'm unclear on if we need to test the Type rep field name lookup and Type rep indexing cases. How often do they happen in a real-world scenario?

Labeled these as low priority in source for now.

Instead of having to uncomment every snippet by hand, can we use CPP to toggle them on and off with a single line change? I plan to rely on HLS (rather than repl) to test it, and I imagine I could just use the hover tooltip to read the error.

Implemented.

[RiugaBachi]

Apologies for that, I probably accidentally deleted the TemplateHaskell pragma at the top and didn't realize it.

[RiugaBachi]

:ok_hand: No prob, take your time. This is understandably a very low priority PR lol. Expanded the TH splice documentation a bit in the meantime.