Closed mrexodia closed 2 years ago
My current guess is that there is no call to FetchContent_Declare
when cached, so likely these functions don't know there is already content for VTIL-Core
? https://gitlab.kitware.com/cmake/cmake/-/blob/f7cf69e34a1607e8ea2b6d10fef6a6058377c24e/Modules/FetchContent.cmake#L651-700
Hey, thanks for the issue!
I agree that the problem seems to be caused by FetchContent
not being aware of the existence of VTIL-Core
, resulting in the dependency being added twice to the project. Currently, you could probably fix the issue by disabling caching for that specific dependency, by adding the argument NO_CACHE true
to the CPMAddPackage
call of VTIL-Core
.
I think it might make sense for CPM.cmake to add another parameter that still allows caching while using FetchContent
to declare packages, to support such use-cases.
For now I indeed disabled CPM caching, but I think this should be considered a bug (the behavior for cache vs no cache should be the same I think). Would it be possible to always use FetchContent
(eg point to the cache directories directly instead of downloading)?
I agree that it would be nice to have FetchContent
and CPM.cmake play nicely together in all circumstances. The reason we have ditched FetchContent
where possible was the drastic performance impact that it brings (sometimes on the order of seconds per dependency when configuring). As projects can easily have hundreds of dependencies, this isn't exactly scalable, so we now use FetchContent
only for the initial download.
However, I haven't yet checked if we are able to use FetchContent_Declare
without running into the performance hit. If so, it would definitely make sense to declare the packages with the correct source directory, as you suggested.
It seems this bug gets to many people and probably should be considered as critical. I am one of them - just found the CPM (I think it's a brilliant idea) and tried to use it in my projects and my first probes gets me to this. I guess I may be one of many. Disabling the cache is not a good workaround as it removes 90% of this module benefits.
@jm4R is still present in the current Version?
@DNKpp yes, but I am not sure if it is exact the same use case. I will describe you mine:
my_project
├── fmtlib
└── labA
└── fmtlib
Where libA
fetches exactly the same version of fmtlib
but using classic FetchContent
After fixing that one by overriding binary directory of direct dependency, another issue appeared:
my_project
├── fmtlib (1)
├── libA
│ └── fmtlib (2)
└── libB
└── fmtlib (3)
This time there is a conflict between (2)
and (3)
. There were no such conflict when libA
and libB
was fetched by classic FetchContent
My guess to fix this would be to override this defaults by some CPM-defaults:
The SOURCE_DIR and BINARY_DIR arguments are supported by ExternalProject_Add(), but different default values are used by FetchContent_Populate(). SOURCE_DIR defaults to ${CMAKE_CURRENT_BINARY_DIR}/
-src and BINARY_DIR defaults to ${CMAKE_CURRENT_BINARY_DIR}/ -build. If a relative path is specified, it will be interpreted as relative to CMAKE_CURRENT_BINARY_DIR.
For example by replacing ${CMAKE_CURRENT_BINARY_DIR}/<lowercaseName>-build
with ${CMAKE_CURRENT_BINARY_DIR}/<lowercaseName>-cpmbuild
. Also it could be an improvement to just use cache path directly for reading source instead of copying it to ${CMAKE_CURRENT_BINARY_DIR}/<lowercaseName>-src
I have the same problem as jm4r mentioned.
FYI: CPM should now be able to override FetchContent calls since v0.35.1! Please me know if you encounter further issues with this.
@DNKpp @TheLartians Just revisited this issue: Problem easily reproduced with 0.38.1 :(
Workaround is wrapping a CPMAddPackage
wit a function:
function(add_dependency depname)
FetchContent_GetProperties(${depname})
string(TOLOWER ${depname} depname_lc)
if(NOT "${${depname_lc}_POPULATED}")
CPMAddPackage(NAME ${depname} ${ARGV})
endif()
endfunction()
And call it:
add_dependency(fmt
URL https://github.com/fmtlib/fmt/releases/download/7.0.3/fmt-7.0.3.zip
URL_HASH SHA256=decfdf9ad274070fa85f26407b816f5a4d82205ae86bac1990be658d0795ea4d
)
BTW: is there a way to revert to the behavior that cached dependencies are copied to <build>/_deps
? Can be helpful when preparing reproducible builds.
In the top-level project there is https://github.com/vtil-project/VTIL-Utils/blob/master/cmake/VTIL-Core.cmake:
And after that it includes https://github.com/vtil-project/VTIL-Utils/blob/master/cmake/VTIL-NativeLifters.cmake:
Inside the
VTIL-NativeLifters
repo there is aFetchContent
call with the nameVTIL-Core
, The idea here is that you can pin a different version ofVTIL-Core
in your project that will then be used to compile theVTIL-NativeLifters
library.https://github.com/vtil-project/VTIL-NativeLifters/blob/master/Dependencies/CMakeLists.txt#L4:
When using
CPM_SOURCE_CACHE
CMake errors out:Without
CPM_SOURCE_CACHE
everything works as intended:This is cmake 3.20.5 btw