Why do doctest `TEST_CASE`s and `SUBCASE`s output to the same table?

[jonas-schulze]

Consider the following example:

#define DOCTEST_CONFIG_IMPLEMENT_WITH_MAIN

#include <nanobench.h>
#include <doctest/doctest.h>

TEST_CASE("foo") {
  double a = 1, b = 2, c;
  auto bench = ankerl::nanobench::Bench();
  SUBCASE("bar") {
    bench.run("+", [&] () { c = a + b; });
  }
  SUBCASE("baz") {
    bench.run("*", [&] () { c = a * b; });
  }
  ankerl::nanobench::doNotOptimizeAway(c);
  CHECK(c == 3);
}

When running the benchmarks without any doctest filters, I would expect to separate tables (two table headers) to be printed. However, there is only one table:

[doctest] doctest version is "2.4.9"
[doctest] run with "--help" for options
Warning, results might be unstable:
* CPU frequency scaling enabled: CPU 0 between 800.0 and 4,900.0 MHz
* CPU governor is 'powersave' but should be 'performance'
* Turbo is enabled, CPU frequency will fluctuate

Recommendations
* Use 'pyperf system tune' before benchmarking. See https://github.com/psf/pyperf

|               ns/op |                op/s |    err% |          ins/op |          cyc/op |         bra/op |   miss% |     total | benchmark
|--------------------:|--------------------:|--------:|----------------:|----------------:|---------------:|--------:|----------:|:----------
|                0.19 |    5,273,674,202.61 |    0.0% |            0.00 |            0.70 |           0.00 |    0.0% |      0.01 | `+`
|                0.19 |    5,273,952,807.67 |    0.0% |            0.00 |            0.70 |           0.00 |    0.0% |      0.01 | `*`
===============================================================================
/home/jschulze/tmp/nanobench#91/test.cpp:6:
TEST CASE:  foo

DEEPEST SUBCASE STACK REACHED (DIFFERENT FROM THE CURRENT ONE):
  baz

/home/jschulze/tmp/nanobench#91/test.cpp:16: ERROR: CHECK( c == 3 ) is NOT correct!
  values: CHECK( 2 == 3 )

===============================================================================
[doctest] test cases: 1 | 0 passed | 1 failed | 0 skipped
[doctest] assertions: 2 | 1 passed | 1 failed |
[doctest] Status: FAILURE!

This is particularly confusing for multiple TEST_CASEs that use the same Bench::name(). Am I having a misconception about how doctest handles test setup and teardown, or about how or when nanobench renders that table?

[martinus]

Doctest has nothing to do with the table output. Use title to start a new table header: https://nanobench.ankerl.com/reference.html#_CPPv4N6ankerl9nanobench5Bench5titleEPKc

[jonas-schulze]

Got it. I thought that two separate Bench objects would always render separate tables, but you only print a new table header if it would be different from the previous one. Smart.

https://github.com/martinus/nanobench/blob/a5a50c2/src/include/nanobench.h#L2335-L2349

I do like the current behavior, I was just afraid that this may be a bug showing through, but it's not. So I can continue using SUBCASEs with nanobench to share the test assertions. (I updated the original example.)