catchorg / Catch2

A modern, C++-native, test framework for unit-tests, TDD and BDD - using C++14, C++17 and later (C++11 support is in v2.x branch, and C++03 on the Catch1.x branch)
https://discord.gg/4CWS9zD
Boost Software License 1.0
18.48k stars 3.02k forks source link

How to get the BDD-style output? #403

Open paolobolzoni opened 9 years ago

paolobolzoni commented 9 years ago

I am probably missing the obvious, but I tried the tutorial BDD example and the output I get is

===============================================================================
All tests passed (16 assertions in 1 test case)

Instead of the described:

Scenario: vectors can be sized and resized
     Given: A vector with some items
      When: more capacity is reserved
      Then: the capacity changes but not the size

How should I get the expected output? Should I pass a specific option to the program? I tried -s, but still it does not look like that at all.

cmaughan commented 9 years ago

Me too...

cmaughan commented 9 years ago

You need to use -s to show successful test output....

paolobolzoni commented 9 years ago

I tried -s, but it does not like the BDD thingy at all.

On Tue, Jun 2, 2015 at 10:05 PM, Chris Maughan notifications@github.com wrote:

You need to use -s to show successful test output....

— Reply to this email directly or view it on GitHub https://github.com/philsquared/Catch/issues/403#issuecomment-107947485.

martinmoene commented 9 years ago

@paolobolzoni You mean you also get the output for the assertions?

paolobolzoni commented 9 years ago

It is possible I misread the documentation, but from the BDD-style section I'd expect the output to be Scenario: vectors can be sized and resized Given: A vector with some items When: more capacity is reserved Then: the capacity changes but not the size

instead I get the normal, non BDD output. So, I guess: yes. It is missing the assertion output.

On Tue, Jun 2, 2015 at 10:12 PM, Martin Moene notifications@github.com wrote:

@paolobolzoni https://github.com/paolobolzoni You mean you also get the output for the assertions?

— Reply to this email directly or view it on GitHub https://github.com/philsquared/Catch/issues/403#issuecomment-107949237.

martinmoene commented 9 years ago

In my test, using cl -EHsc main.cpp && main.exe -s. the output for the last section is:

prompt>cl -EHsc main.cpp && main.exe -s
. . .
-------------------------------------------------------------------------------
Scenario: vectors can be sized and resized
     Given: A vector with some items
      When: less capacity is reserved
      Then: neither size nor capacity are changed
-------------------------------------------------------------------------------
main.cpp(6)
...............................................................................

main.cpp(42):
PASSED:
  REQUIRE( v.size() == 5 )
with expansion:
  5 == 5

main.cpp(43):
PASSED:
  REQUIRE( v.capacity() >= 5 )
with expansion:
  5 >= 5

===============================================================================
All tests passed (16 assertions in 1 test case)

prompt>
brinkrob commented 9 years ago

It would be interesting to get the BDD style descriptions as a sort of test report, without having all the successful REQUIRE statements printed out since that can add a lot of noise.

martinmoene commented 9 years ago

(You can get assertions-only by using the compact reporter ;)

prompt>main --reporter compact --success
main.cpp(11): passed: v.size() == 5 for: 5 == 5
main.cpp(12): passed: v.capacity() >= 5 for: 5 >= 5
main.cpp(18): passed: v.size() == 10 for: 10 == 10
main.cpp(19): passed: v.capacity() >= 10 for: 10 >= 10
main.cpp(11): passed: v.size() == 5 for: 5 == 5
main.cpp(12): passed: v.capacity() >= 5 for: 5 >= 5
main.cpp(26): passed: v.size() == 0 for: 0 == 0
main.cpp(27): passed: v.capacity() >= 5 for: 5 >= 5
main.cpp(11): passed: v.size() == 5 for: 5 == 5
main.cpp(12): passed: v.capacity() >= 5 for: 5 >= 5
main.cpp(34): passed: v.size() == 5 for: 5 == 5
main.cpp(35): passed: v.capacity() >= 10 for: 10 >= 10
main.cpp(11): passed: v.size() == 5 for: 5 == 5
main.cpp(12): passed: v.capacity() >= 5 for: 5 >= 5
main.cpp(42): passed: v.size() == 5 for: 5 == 5
main.cpp(43): passed: v.capacity() >= 5 for: 5 >= 5
Passed 1 test case with 16 assertions.
itrion commented 9 years ago

Thank you very much for this tip Martin :D

On Tue, 2 Jun 2015 at 21:00 Martin Moene notifications@github.com wrote:

(You can get assertions-only by using the compact reporter ;)

main.cpp(11): passed: v.size() == 5 for: 5 == 5 main.cpp(12): passed: v.capacity() >= 5 for: 5 >= 5 main.cpp(18): passed: v.size() == 10 for: 10 == 10 main.cpp(19): passed: v.capacity() >= 10 for: 10 >= 10 main.cpp(11): passed: v.size() == 5 for: 5 == 5 main.cpp(12): passed: v.capacity() >= 5 for: 5 >= 5 main.cpp(26): passed: v.size() == 0 for: 0 == 0 main.cpp(27): passed: v.capacity() >= 5 for: 5 >= 5 main.cpp(11): passed: v.size() == 5 for: 5 == 5 main.cpp(12): passed: v.capacity() >= 5 for: 5 >= 5 main.cpp(34): passed: v.size() == 5 for: 5 == 5 main.cpp(35): passed: v.capacity() >= 10 for: 10 >= 10 main.cpp(11): passed: v.size() == 5 for: 5 == 5 main.cpp(12): passed: v.capacity() >= 5 for: 5 >= 5 main.cpp(42): passed: v.size() == 5 for: 5 == 5 main.cpp(43): passed: v.capacity() >= 5 for: 5 >= 5 Passed 1 test case with 16 assertions.```

— Reply to this email directly or view it on GitHub https://github.com/philsquared/Catch/issues/403#issuecomment-108079113.

torbjoernk commented 8 years ago

@martinmoene How does your tip back from 2015 (sorry for this issue necromancy) help in only getting the BDD scenario output without the assertions?

Followup question: Given I have an assertion in a WHEN but outside a THEN block, how can I suppress the first occurrence of the BDD-style output block (the one without the Then:)?

SCENARIO("Add or Multiply") {
  GIVE("two values") {
    int a = 1, b;
    WHEN("both are the same") {
      b = a;
      REQUIRE(a == b);
      THEN("the sum is the same as multiply the first by 2") {
        REQUIRE(a + b == a * 2);
      }
    }
  }
}
$> ./foo_test -s -r console
foo_test is a Catch v1.5.6 host application.
Run with -? for options

-------------------------------------------------------------------------------
Scenario: Add or Multiply
     Given: two values
      When: both are the same
-------------------------------------------------------------------------------
/$PROJECTS/foo/tests/foo_test.cpp:9
...............................................................................

/$PROJECTS/foo/tests/foo_test.cpp:17: 
PASSED:
  REQUIRE( a == b )
with expansion:
  1 == 1

-------------------------------------------------------------------------------
Scenario: Awesomeness
     Given: two values
      When: both are the same
      Then: the sum is the same as multiply the first by 2
-------------------------------------------------------------------------------
/$PROJECTS/foo/tests/foo_test.cpp:9
...............................................................................

/$PROJECTS/foo/tests/foo_test.cpp:22: 
PASSED:
  REQUIRE( a + b == a * 2  )
with expansion:
  true

I only want the last Scenario block with the then line.

martinmoene commented 8 years ago

@martinmoene How does your tip back from 2015 (sorry for this issue necromancy) help in only getting the BDD scenario output without the assertions?

@torbjoernk It doesn't help, hence the parenthetical remark.

kakyoism commented 5 years ago

(You can get assertions-only by using the compact reporter ;)

prompt>main --reporter compact --success
main.cpp(11): passed: v.size() == 5 for: 5 == 5
main.cpp(12): passed: v.capacity() >= 5 for: 5 >= 5
main.cpp(18): passed: v.size() == 10 for: 10 == 10
main.cpp(19): passed: v.capacity() >= 10 for: 10 >= 10
main.cpp(11): passed: v.size() == 5 for: 5 == 5
main.cpp(12): passed: v.capacity() >= 5 for: 5 >= 5
main.cpp(26): passed: v.size() == 0 for: 0 == 0
main.cpp(27): passed: v.capacity() >= 5 for: 5 >= 5
main.cpp(11): passed: v.size() == 5 for: 5 == 5
main.cpp(12): passed: v.capacity() >= 5 for: 5 >= 5
main.cpp(34): passed: v.size() == 5 for: 5 == 5
main.cpp(35): passed: v.capacity() >= 10 for: 10 >= 10
main.cpp(11): passed: v.size() == 5 for: 5 == 5
main.cpp(12): passed: v.capacity() >= 5 for: 5 >= 5
main.cpp(42): passed: v.size() == 5 for: 5 == 5
main.cpp(43): passed: v.capacity() >= 5 for: 5 >= 5
Passed 1 test case with 16 assertions.

This is nice, but doesn't it hide the bdd report as well? Could we see the bdd report PLUS this?