[question] Conan - CMake package development workflow: unit-testing

mpll-d commented 8 months ago

What is your question?

Dear Conan Team,

As far as I understood, the unit-testing setup described in Conan help requires to create a separate Conan package to host the tests, that depends on the lib one wants to test. This works well but does not allow for a smooth package development workflow.

What I want to do

I would like to be able to use CMake integration in IDEs (VSCode/VS2022) in order to smoothly do the cycle write code - build - test, without having to go back to any Conan call from the command line.

In other words, I'd like to:

do a conan install ...
Work only on writing code and calling CMake configure/build/test
Once I'm happy, I push my changes and let my CI pipeline take care of the conan create & conan update

Why I am stuck

Finding the built shared objects at tests-runtime

As recommended in the documentation, in my conanfile.py, the build() step definition is minimal (just cmake.configure() & cmake.build() calls) and all the setup is made in the generate() function.

The Lib I want to build is a shared lib (let's call it thelib), and I want to test it with CTest & GTest. Conan provides then the generators/CMakePrestets.json file with a testPreset that sets the runenv info of the packages my lib depends on.

Since the environment provided in the testPreset does not contain the path to the lib runtime location (thelib.so or thelib.dll), running the test fails. What is the proper way to provide the proper runtime location, so that CTest runs finds thelib.so? The layout function specifies the location of the libs, maybe it's possible to use this then?

Finding non-conan-related 3rd party libs

Assuming that I want to be able to specify the location of an external runtime library that is not handled by Conan and append/prepend it to the LD_LIBRARY_PATH that Conan will use. I would like to have it in an own CMakeUserPresets.json. This is possible to set up with tc.user_presets_path setting. Unfortunately, if I inherit Conan's testPreset and set LD_LIBRARY_PATH, this will overwrite (and not extend) the Conan-provided LD_LIBRARY_PATH.

Is there a way to specify a prefix/suffix for CMakeToolchain to generate a generators/CMakePresets.json that has prefixed/suffixed environment variables? For example .../generators/CMakePresets.json

    "testPresets": [
        {
            "name": "conan-relwithdebinfo",
            "configurePreset": "conan-relwithdebinfo",
            "environment": {
                "LD_LIBRARY_PATH_CONAN": "...",
                "PATH_CONAN": "..."
            }
        }
    ]

This way I can have in my own /CMakeUserPresets.json something like:

    "testPresets": [
        {
            "inherits": "conan-relwithdebinfo",
            "name": "conan-extended-ldpath",
            "environment": {
                "LD_LIBRARY_PATH": "some-3rdparty-system-lib/lib:$penv{LD_LIBRARY_PATH_CONAN}:$penv{LD_LIBRARY_PATH}"
            }
        }
    ]

Or, is there just a better built-in way than this suffixing idea?

Sidenotes

Debugging tests

Debugging tests - on VSCode - cannot rely on CMakePresets. It requires a launch.json that may use a .env file. It may be nice to have a generator that creates such an .env file. Note that this should contain both the dependencies runtime locations and the thelib.so location.

Sidenote on VS2022

VS2022 seems to not well integrate CMakePresets for tests (the test explorer UI does not provide the ability to pick a test preset so tests are "just" run in a console instead of the more readable test explorer display). Maybe it could help to provide a generator that creates a .runsettings file specifying the environment variables for the tests, to be used in place of the testPreset.

Thanks for your help and the great work so far with Conan, Any suggestion of a better way to work with Conan / CMake / CTest is most welcome, don't hesitate to let me know if I'm having anti-patterns that I can eliminate

Best, M.

Have you read the CONTRIBUTING guide?

[X] I've read the CONTRIBUTING guide

memsharded commented 8 months ago

Hi @mpll-d

Thanks for your question

As far as I understood, the unit-testing setup described in Conan help requires to create a separate Conan package to host the tests, that depends on the lib one wants to test. This works well but does not allow for a smooth package development workflow.

I am afraid there is some kind of misconception, or probably some flaw in our docs. Unit testing cannot be in general be external to the package, because only the external public interfaces are exposed, which are usually not enough for unit testing. The recommended unit-test is something in build() method:

def build(self):
      cmake = CMake(self)
      cmake.configure()
      cmake.build()
      cmake.test()  # this are unit tests

The tools.build:skip_test conf can disable running the tests for faster builds.

The external test_package functionality is not for running unit-tests, but for testing the integrity of the package and capture early possible packaging mistakes.

I would like to be able to use CMake integration in IDEs (VSCode/VS2022) in order to smoothly do the cycle write code - build - test, without having to go back to any Conan call from the command line.

This is a totally valid legit flow, and Conan should be able to provide it.

Lets go one issue at a time, lets see if I understood it correctly:

The Lib I want to build is a shared lib (let's call it thelib), and I want to test it with CTest & GTest. Conan provides then the generators/CMakePrestets.json file with a testPreset that sets the runenv info of the packages my lib depends on.

Since the environment provided in the testPreset does not contain the path to the lib runtime location (thelib.so or thelib.dll), running the test fails. What is the proper way to provide the proper runtime location, so that CTest runs finds thelib.so? The layout function specifies the location of the libs, maybe it's possible to use this then?

Is the thelib.so a dependency library? or the library that is being built at this package. Because Conan can only manage the path to the dependencies, when you are executing conan install, only dependencies exist. The current project targets, libraries are completly unknown, until you execute cmake, but that part Conan is completely unaware and not called anymore. So the path to your own project thelib.so must be handled by your own CMakeLists.txt.

Is this the case?

mpll-d commented 8 months ago

Hi @memsharded,

Thanks for your swift answer.

Is the thelib.so a dependency library?

No, it is indeed the lib that the package is building. Thanks for pointing it out: as I'm reading you it becomes clear that this is not something that could be handled (nor should be handled) by the CMakePresets.json, so I need to find out how to make CMake itself aware of the target's runtime thelib.so / thelib.dll location.

memsharded commented 8 months ago

No, it is indeed the lib that the package is building. Thanks for pointing it out: as I'm reading you it becomes clear that this is not something that could be handled (nor should be handled) by the CMakePresets.json, so I need to find out how to make CMake itself aware of the target's runtime thelib.so / thelib.dll location.

Yes, I think so. The thing is that I have also approached/tried this a few times in the past, and I cannot recall CMake having an elegant solution for this. For .so isn't generally an issue, as local executables rpaths can work to find the .so, while .dll can be a pain, and I have seen CMake users doing bare copies (`with cmake custom commands) of the dlls, to put them besides the test executables.

conan-io / conan