SETUP & SECTION reporting

martinmoene / lest

A modern, C++11-native, single-file header-only, tiny framework for unit-tests, TDD and BDD (includes C++98 variant)

Boost Software License 1.0

390 stars 45 forks source link

SETUP & SECTION reporting #61

Closed beatrate closed 6 years ago

beatrate commented 6 years ago

It would be beneficial to have the SECTION names be reported with their success status. As it is now, their names are silently ignored and all asserts are just printed with the title of the outermost CASE. I think it would bring more granular control of tests where we can see the failing subtest's goal right away.

I suggest reporting them like this

1>main.cpp(39): Ill formed URLs are rejected
1>main.cpp(40): ====passed: Empty URL is rejected: FailParse("") for true
1>main.cpp(50): ====passed: URL with invalid protocol is rejected: FailParse("htt://some.org") for true

Where the CASE is "Ill formed URLs are rejected" and the rest are SECTIONs inside the case's setup. This format could be a special verbose mode activated with the corresponding --verbose parameter.

martinmoene commented 6 years ago

It would be beneficial to have the SECTION names be reported with their success status.

I do agree.

There's a longstanding PR on this subject: Added reporting of intermediate description text from nested CASE, SETUP, and SECTIONs (PR #36).

lest is intended to be small and relatively simple, attributes I'm keen to keep (see e.g. the way sections are handled). I've yet to find a way to add reporting of sections in a manner that fits lest.

martinmoene commented 6 years ago

Perhaps, I should take a look at PR #36 again.

martinmoene commented 6 years ago

Implement report section on failure inspired on PR #36:

[x] Add option-v, --verbose to also report sections on failure (keep default backwards compatible)
[x] Update Readme for option -v, --verbose
[x] Add code to report sections on failure (lest.hpp)
[x] Add code to report sections on success (lest.hpp)
[x] Add code to report sections on failure (lest_cpp03.hpp)
[x] Add code to report sections on success (lest_cpp03..hpp)
[x] Add test(s) for reporting of sections (test_lest..hpp)): Option -v,--verbose also report passing or failing sections [commandline]
[x] Add test(s) for reporting of sections (test_lest_cpp03..hpp)): Option -v,--verbose also report passing or failing sections [commandline]
[x] ~Try and prevent excessive copying of strings~
[x] Non-goal: ~Add code to report sections in listing of tests~