redding / assert

Assertion style testing framework.
https://github.com/redding/assert
MIT License
10 stars 1 forks source link

verbose output mode #210

Closed kellyredding closed 9 years ago

kellyredding commented 9 years ago

This adds a verbose runtime output mode flag to the CLI and config. This also adds a method for querying this flag to the common view helpers. Finally, this updates the default view to honor the flag in its output.

This goal here is to be able to give more details about what is running at runtime.

Closes #208.

@jcredding ready for review

kellyredding commented 9 years ago

Example output from assert's test suite:

$ assert --help
Usage: assert [options] [TESTS]

Options:
    -s, --runner-seed RUNNER-SEED    Use a given seed to run tests
    -o, --[no-]capture-output        capture stdout and display in result details
    -t, --[no-]halt-on-fail          halt a test when it fails
    -c, --[no-]changed-only          only run test files with changes
    -p, --[no-]pp-objects            pretty-print objects in fail messages
    -e, --[no-]profile               output test profile info
    -v, --[no-]verbose               output verbose runtime test info
    -d, --[no-]debug                 run in debug mode
        --version
        --help
$ assert -v
Loaded suite (445 tests)
Running tests in random order, seeded with "50696"
"Assert::Context should respond to methods" (Assert::Assertions::UnitTests)
    /Users/kelly/projects/redding/assert/test/unit/assertions_tests.rb:15
    ................................................... 0.004369 seconds, 51 results, 11673.151751 results/s
"`assert_nothing_raised` should produce results as expected" (Assert::Assertions::AssertNothingRaisedTests)
    /Users/kelly/projects/redding/assert/test/unit/assertions/assert_raises_tests.rb:55
    ... 0.001290 seconds, 3 results, 2325.581395 results/s
"`assert_not_nil` should produce results as expected" (Assert::Assertions::AssertNotNilTests)
    /Users/kelly/projects/redding/assert/test/unit/assertions/assert_nil_tests.rb:49
    ... 0.001960 seconds, 3 results, 1530.612245 results/s
"Assert::Stub for singleton methods on a module should allow stubbing a method that takes any args" (Assert::Stub::ModuleTests)
    /Users/kelly/projects/redding/assert/test/system/stub_tests.rb:189
    ... 0.001204 seconds, 3 results, 2491.694352 results/s
"`assert_not_true` should have a fail message with custom and generic explanations" (Assert::Assertions::AssertNotTrueTests)
    /Users/kelly/projects/redding/assert/test/unit/assertions/assert_true_false_tests.rb:55
    . 0.000497 seconds, 1 results, 2012.072435 results/s
[...]
jcredding commented 9 years ago

@kellyredding - Looks good :boom:

kellyredding commented 9 years ago

@jcredding I'm thinking I should add the file and line number for each test to this output (after I merge #211). What do you think? Too noisy? Where should it go? On the line with the full test name and context class? Replace the context class? New line?

Would love your thoughts on this. Thanks!

jcredding commented 9 years ago

@kellyredding - It would be noisy but that's kinda the point of verbose mode. If you wanted to keep this as-is you could do the -vv which is like very verbose or something. I've seen that a few times. I'd be fine adding the path and line number to this though cause like I said, it's verbose.

As far as where, I'd probably put it on it's own line after the description and context class. I like the dots and the stats being the last line (mirrors the full test suite results in non-verbose mode) and the description and context class being the first (this is what I would be looking for to identify the test). I like the new line because if its a file path and line number its going to probably be longer and having a full line to read it seems easier. It's harder to read a file path If it gets wrapped.

kellyredding commented 9 years ago

@jcredding good thoughts - thanks for the input. That's kinda how I was leaning too. I'll merge #211 and then update this. Thanks man!

kellyredding commented 9 years ago

@jcredding I've update this after rebasing off #211 and #213. It now shows the file line info in the verbose output. I've updated my comment above to reflect this. I also added a README snippet for this option. Please review the output tweaks and the README snippet. Thanks!

jcredding commented 9 years ago

@kellyredding - I like the indention, helps quickly find the test name. The readme snippet looks good and makes sense :boom: