catch_discover_tests doesn't use the same environment to discover tests through TEST_EXECUTABLE

[iomaganaris]

Describe the bug Using cmake to build my application and catch_discover_tests to find all the catch tests defined, I came across the following issue. While it's possible to set PROPERTIES in catch_discover_tests that will be used to run the tests with, those same properties are not passed to the process that runs the test executable to parse the various tests defined in the application.

Expected behavior I would expect that the same environment would be used to run the test executable to find out which tests to run and to actually run the tests.

Reproduction steps catch_discover_tests(${test_name} TEST_PREFIX "${test_name}/" PROPERTIES ENVIRONMENT "TEST_ENV_VAR=MY_VAR")

Platform information:

• OS: Linux
• Compiler+version: GCC v7.4.0,8.2.0/clang 8
• Catch version: master brach
  • Cmake version: 3.10.2

Supplementary information: I would like to be able to set one or more environment variables to a specific value for the tests only, without having to manually set it through the command line before compiling the application to run every invocation of the application executable with the same environment.

[mvainioe]

I ran into the same issue, the tests discovery fails to load a shared library not found. Would need to set LD_LIBRARY_PATH for both discovery and running the tests.

[mxmlnkn]

I have the same problem. Tests were not found on a headless server because QT_QPA_PLATFORM=offscreen was not set for test discovery. My current workaround is to export it inside a custom test catch main.

[mxmlnkn]

I had a similar problem again wondering why test discovery failed with program exited with code 0xc0000139 on Windows after upgrading to Qt6 while it worked with Qt5. It turns out that it used some completely unrelated Qt5 dlls randomly being available in the PATH while the actual Qt installations were not used. The issue is again that environment variables are seemingly not propagated to test discovery. Why not just propagate all environment variables?

I was able to fix that problem by using DL_PATH, which is a partial solution to this whole conundrum.

catch_discover_tests(${test}
    PROPERTIES ENVIRONMENT "${_environment_vars_list}"
    DL_PATHS "$ENV{PATH}" "$ENV{LD_LIBRARY_PATH}")

Edit: I was wrong. It doesn't work yet for some reason.

[alkino]

PROPERTIES apply on a cmake target. For getting the list of tests the cmake function execute_process is used that does not support PROPERTIES. To have env variable it will needs to parse properties and use a mechanism (such as https://stackoverflow.com/questions/62927946/passing-environment-variable-to-the-command-in-cmake-execute-process#62935259) to work. Today, this is not possible (nothing done for that in https://github.com/catchorg/Catch2/blob/devel/extras/CatchAddTests.cmake#L63-L68), so what next?