tcbrindle / cpp17_headers

C++17 library facilities for older compilers
145 stars 27 forks source link

Increased testing coverage #6

Open madscientist opened 7 years ago

madscientist commented 7 years ago

It would be really great if we could expand the test coverage of the package to test more combinations of things: for example, if we can build/run tests with C++14 support, and with extensions enabled and disabled, and test both with (as now) and without forcing the internal versions to be used so we can verify the support for the native <experimental/*> headers when they're available.

The latter can be tested by just building the test files multiple times with different -D options to enable/disable the internal versions, but building with different levels of C++ support and enable/disable extensions might require multiple invocations of cmake since it doesn't seem you can just reset these values at build time... gross. Maybe adding some info to the README or including a script that will use out-of-source-tree builds to run cmake/compile multiple ways.

Also, it would be nice if we can enhance the test main() methods to show some info about which model we used to compile, e.g. whether the test used the native implementation vs. the stx implementation.

tcbrindle commented 7 years ago

Agreed.

I tried doing it, so there were four versions of most tests (two for variant); one with C++11, one with C++11 and -DSTX_NO_BLAH (via CMake's target_compile_definitions()), and the same again in C++14 mode. The problem is that test_optional.cpp in particular assumes that it's testing its own implementation and has lots of build failures with std::experimental::optional from libstdc++, so those parts would need to be chopped out.

madscientist commented 7 years ago

Ah, I didn't realize that test_optional was testing more of the internal implementation rather than just the published API. Yeah, that would need to be separated out and put behind some ifdefs I guess.

tcbrindle commented 7 years ago

As you can see from the commit, I've tried again to test all four possibilities (C++11/14, with/without overrides) using some CMake hackery. To address the issue of test_optional.cpp and test_any.cpp testing just the specific implementation, I've added to very simple new files, test_any_basic.cpp and test_optional_basic.cpp -- the idea is just to make sure we can create the required type and link the resulting executable. This should help avoid the situation with std::experimental::optional on libc++ turning up again.

(It also helped me discover that libstdc++'s any, optional and string_view don't work in C++11 mode, so I've forced overrides in that case.)

Do you think that covers it for now, or is there something obvious I'm missing?