Misc feature requests

[af]

Hi,

I've used Mocha, tape, and AVA in the past, but painless seems to hit the intersection of features that's just right for me :+1: That said, after switching one of my projects over there were a few small things I missed and features that I think would be nice additions. I didn't want to spam the repo with an issue for each item since you might not be interested in adding all (any?) of them:

• Show total number of tests passed/failed at end of test output (eg. "100 passed, 2 failed")
• Show execution time, both for each test, and for the test suite total
• add a string name parameter for createGroup(), which could be displayed in the output before the group's tests
• expose chai.use() for adding new plugins
• Standard reporter API for creating new reporters, and CLI option for selecting them
• A few more included reporters (mocha's dot reporter is a nice minimalistic one that I use often)
• Add an option to bail after the first failed test (like mocha's --bail)

Since it looks like the tap-spec package is currently doing the default formatting/reporting, some of these might be difficult without forking/replacing it.

Just my 2¢, hope the feedback is useful!

[taylorhakes]

Thanks for the feedback!. All of your suggestions are in line with what I was thinking. I will work on adding all of them. The spec reporter that I am using currently doesn't support test number or execution time. I am working on creating a new spec reporter for that reason.

You do have access to TAP output though. You can use any reporter that supports TAP. Here is a list reporters on the Tape readme. Unfortunately, not all the reporters adhere to the spec and might be missing output when used with painless. I will create a list of officially supported reporters.

If you run painless with the -t or --tap command line arg it will output TAP. It will show you test number and time.

You can use the tap-dot reporter using the tap output from painless. Here is the way.

npm install tap-dot --save-dev

Then change your package.json

{
  "name": "module-name",
  "scripts": {
    "test": "painless ./test/**/*.js --tap | tap-dot"
  }
}

Or via the command line

./node_modules/.bin/painless ./test/**/*.js --tap | ./node_modules/.bin/tap-dot

I am not a fan of that dot reporter output though. It doesn't show the full error when a test fails. I will create a new dot reporter as well.

[af]

Awesome, excited to hear that a lot of these things are on the roadmap! One more thing I forgot to mention:

• Flag slow tests (with an option like mocha's --slow)

It's a good point that all of the 3rd party TAP reporters will work with painless already. For something as simple as the dot reporter, that should be good enough in most cases. At the same time, having things like slow test reporting is pretty nice and you can't do it with stock tap output.

[taylorhakes]

Definitely. The TAP output format is very limiting. It doesn't support groups, times, etc.

Under the hood painless uses a JSON object stream reporter that is piped to TAP and then piped to the spec reporter. The object stream is a lot easier to consume and supports more info than TAP. I will be building the new spec and dot reporter off of that. I will probably expose that stream for custom reporters to use.

You will also always have TAP available as well, if you want to use one of the TAP reporters.

[taylorhakes]

I created a roadmap for 0.8.0. Please feel free to continue discussion there.

https://github.com/taylorhakes/painless/issues/10