search

TheSeamau5 / elm-check

Property Based Testing in Elm
70 stars 20 forks source link

Working towards an extensible front-end/output format #23

Open TheSeamau5 opened 9 years ago

TheSeamau5 commented 9 years ago

It seems that there are a number of things one wants to do with tests:

  1. Group tests and test suites into suites
  2. Run all tests at once
  3. Run individual tests à la carte
  4. View test results as JSON
  5. View test results as HTML
  6. View tests in an interactive runner (browser/console) where one can request more tests/manual provide data from individual claims
  7. Export tests as elm-test assertions

I currently have a hard time figuring out how one could potentialy support all of these at once with the same API (without the Check/Test distinction).

@rtfeldman, given you are a user of elm-check and seem pretty knowledgeable about testing frameworks, I'd like to bring you in to this discussion.

The goal here is to come up with an API that makes most sense from a practical perspective.

rtfeldman commented 9 years ago

Agree with these, except that "View test results as JSON" really wants to be "view test results as JUnit XML format." That's what CI servers (we use Jenkins at work) tend to accept in order to report nicely on individual test failures instead of just spitting out console output.

The last three (view test results as HTML, view tests as an interactive runner, export tests as elm-test assertions) are very low priority for me. Running individual tests a la carte is a big deal, but that's really a CLI change.