"requiredness" of find_package dependencies not modifiable by super-projects

[lizziel]

Several dependencies are hard-coded as REQUIRED in calls to find_package. This causes CMake configure problems when pFlogger is built in a super-project that does not use GMAO Baselibs. https://github.com/Goddard-Fortran-Ecosystem/pFlogger/blob/25c3c6eddaf20bf26d4cbd5c00f2317bcadb5f7b/CMakeLists.txt#L36-L38

The find_package calls can be made more flexible by using an IS_REQUIRED variable set at a higher level for whether the package must be found. For example:

find_package(GFTL ${GFTL_IS_REQUIRED_ARG} CONFIG)
find_package(GFTL_SHARED ${GFTL_SHARED_IS_REQUIRED_ARG} CONFIG)
find_package(YAFYAML ${YAFYAML_IS_REQUIRED_ARG} CONFIG) 

This issue is also relevant to find_package calls within CMakeLists.txt of MAPL and yaFyaml.

[mathomp4]

@lizziel I'm a bit confused. To build pFlogger they are required. Are you somehow building pFlogger without them?

[tclune]

I've explained to @mathomp4 via backchannels.

[tclune]

Thinking about it some more, I should add the explanation in this thread for others that might need to understand.

If a codebase uses submodules to directly include the pFlogger dependencies then the CMake find_package() is unnecessary as CMake will find the relevant target automatically. However find_package(... REQUIRED) will fail as it does not recognize internal targets. (Good reasons for that.) So some solution is needed to enable builds of that sort.

The only options that I am currently aware of are (1) putting in some sort of conditional as alluded to by the OP, or (2) using CMake's ExternalProject command. This command can download (or clone) external dependencies and build&install. In this scenario the download step would be skipped.

@lizziel Let me know if you have the energy to try ExternalProject. I'm thinking this might be useful in your environment as it gives you direct control over how these dependencies are handled.

[lizziel]

ExternalProject may indeed be the best long-term solution for us. @LiamBindle mentioned this is something he could look more into later in the summer.

[LiamBindle]

Hi everyone. I can take a look at ExternalProject again (later in the summer once I get my thesis done). I looked into it in the past for GEOS-Chem, and I decided against it, so I am a bit wary of ExternalProjects. The reason I ruled it out in the past is (IIRC) ExternalProjects configures and builds the project during the configure step of the super project (i.e. during the cmake step not the make step). If the ExternalProject's configure/build isn't airtight, build errors/warnings show up in the cmake step, and my concern is that this might be confusing or alarming to our users that aren't familiar with CMake. My evaluation was based on making the build as simple and seamless as possible from our users perspective, because in the past compiling ESMF, MAPL, and GCHP has been a barrier for new users. That being said, I was considering it as a possibility for building MAPL (and/or) ESMF though, so I suspect ExternalProjects would be more appropriate for small libraries like gFTL and the other utility libraries.

Are there problems that come up by skipping the find_package() call, or using the actual target rather than the imported target? The reason I like this approach is super project can disable the find_package() call by (1) setting the requiredness to "" and then (2) setting CMAKE_DISABLE_FIND_PACKAGE_GFTL to TRUE, but it doesn't affect the standalone project. A similar approach would be something like

if (not target GFTL::GFTL)
    find_package(GFTL REQUIRED)
endif()

Here, if the GFTL::GFTL target is already defined (e.g. by a previous add_subdirectory()), the find_package() call is skipped.

[tclune]

@LiamBindle that was my concern as well. I thought I recently ran across a variant that defers the build until, er, build. But my memory is such that I don't recall the name of the feature. I'll try poking around to see if I can find it in an email from the person that I think mentioned it to me.

Lacking that, I agree that a conditional is the only option. I find the second variant with an explicit conditional easier to understand, but only marginally.

[LiamBindle]

@tclune That would be great for us if there was a way to defer their builds until after the build step. In fact, if that was the case, I think GCHPctm could potentially build MAPL and all its dependencies with that approach. I don't see why CMake couldn't do this--all the info that goes into a ConfigModule is knowable during the config step--at least, I think it is.

To iterate on that explicit conditional, perhaps something like

set(PFLOGGER_RESOLVE_DEPENDENCIES_FROM_CURRENT_SCOPE "FALSE" CACHE ...)
if (not PFLOGGER_RESOLVE_DEPENDENCIES_FROM_CURRENT_SCOPE)
   find_package(GFTL REQUIRED)
endif()

would be more clear.