msis commented 5 years ago

I can't seem to be able to add soci as a subproject.

I have tried with ExternalProject_Add, and FetchContent but I can't figure out why I keep getting:

CMake Error at src/test_app/CMakeLists.txt:21 (add_dependencies):
  The dependency target "soci" of target "test_app" does not exist.

Project description

My project structure is the following:

└── src
    ├── CMakeLists.txt
    ├── cmake
    │   ├── soci.cmake
    │   └── spdlog.cmake
    ├── test_app
    │   └── CMakeLists.txt
    └── (...)

`src/CMakeLists.txt`

cmake_minimum_required(VERSION 3.11)

set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)

include(cmake/spdlog.cmake)
include(cmake/soci.cmake)

add_subdirectory(test_app)

`src/test_app/CMakeLists.txt`

cmake_minimum_required(VERSION 3.11)

project(test_app LANGUAGES CXX)

add_executable(test_app main.cpp)

target_link_libraries(test_app PRIVATE
    spdlog
    SOCI::core
)

add_dependencies(test_app spdlog soci)

`spdlog.cmake`

cmake_minimum_required(VERSION 3.11)

message(STATUS "Extern: spdlog v1.4.2")

set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)

include(FetchContent)

FetchContent_Declare( spdlog
    GIT_REPOSITORY https://github.com/gabime/spdlog
    GIT_TAG v1.4.2
    GIT_SHALLOW ON
)
FetchContent_GetProperties(spdlog)
if(NOT spdlog_POPULATED)
    set(SPDLOG_BUILD_EXAMPLE OFF)
    set(SPDLOG_BUILD_EXAMPLE_HO OFF)
    set(SPDLOG_BUILD_TESTS OFF)
    set(SPDLOG_BUILD_TESTS_HO OFF)
    set(SPDLOG_BUILD_BENCH OFF)
    set(SPDLOG_SANITIZE_ADDRESS OFF)
    set(SPDLOG_INSTALL ON)
    set(SPDLOG_FMT_EXTERNAL OFF)
    FetchContent_Populate(spdlog)
    add_subdirectory(
        ${spdlog_SOURCE_DIR}
        ${spdlog_BINARY_DIR}
        EXCLUDE_FROM_ALL)
endif()

`soci.cmake`

cmake_minimum_required(VERSION 3.11)

message(STATUS "Extern: SOCI pre4.0+git_blksail")

set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)

include(FetchContent)

FetchContent_Declare( soci
    GIT_REPOSITORY https://github.com/SOCI/soci
    GIT_TAG master
    GIT_SHALLOW ON
)
FetchContent_GetProperties(soci)
if(NOT soci_POPULATED)
    set(SOCI_STATIC ON)
    set(SOCI_SHARED ON)
    set(SOCI_TESTS OFF)
    # set(SOCI_ASAN OFF)
    set(SOCI_CXX11 ON)
    set(SOCI_LIBDIR lib)
    set(WITH_SQLITE3 ON)
    set(WITH_POSTGRESQL ON)
    set(WITH_BOOST OFF)
    set(WITH_DB2 OFF)
    set(WITH_ODBC OFF)
    set(WITH_ORACLE OFF)
    set(WITH_MYSQL OFF)
    set(SOCI_EMPTY OFF)
    FetchContent_Populate(soci)
    add_subdirectory(
        ${soci_SOURCE_DIR}
        ${soci_BINARY_DIR}
        EXCLUDE_FROM_ALL)
endif()

mloskot commented 5 years ago

I'm sorry but I can't help you. I almost never use dependencies/projects via ExternalProject_Add

msis commented 5 years ago

Thanks @mloskot . I believe it has to do with a missing export in SOCI's CMakeLists.txt. I will keep digging.

tt4g commented 5 years ago

@msis For your information.

Although it is a little complicated, it is important to understand that CMake operations are roughly divided into two steps: configuration and build.

FetchContent_Declare() and add_dependencies() are executed in configure step.

At this point, soci is not built by CMake (soci is a library that needs to be built), because the configure step only downloads the source. If it is not built and installed, find_package(soci) cannot find the soci CMake target.

NOTE: That's because FetchContent_Declare() and ExternalProject_Add() are supposed to get the source code necessary for your project from outside. It is possible to obtain the source of the third party library, but the build work is performed in the CMake build step: FetchContent — CMake 3.16.0-rc2 Documentation

To build SOCI with CMake alone and use it in your project, you need to build the SOCI with CMake configure step and define its CMake target. This requires complex CMake operations, but there are googletest samples.

In the googletest sample below, execute_process() is executed by CMake configure step, and the cmake command is called to execute googletest build. googletest built with add_subdirectory () can be referenced in the project.

https://github.com/google/googletest/blob/master/googletest/README.md

Since spdlog is a header-only-library, it seems to be working.

msis commented 5 years ago

Thanks @tt4g. I will try what you suggested right away.

However, I added a static library that worked too. I see it build, and I suppose it's because of the add_dependencies(). And it doesn't complain that the dependency target does not exist.

PS: So far it seems that all other libraries build a package with include(CMakePackageConfigHelpers) and soci doesn't.

msis commented 5 years ago

@tt4g , I followed the steps for googletest. I see it compile, but when it is time to build test_app, I do still have the same error:

CMake Error at src/test_app/CMakeLists.txt:19 (add_dependencies):
  The dependency target "soci" of target "test_app" does not exist.

tt4g commented 5 years ago

soci's CMake target name is soci_core. The backend library must also specify each CMake target (e.g. soci_${backend}).

Try replacing the dependency name soci with soci_core.

msis commented 5 years ago

We're making progress! It now compiles soci with the new dependency name.

But it seems like there's another issue with the include directories: The following will not compile:

#include "soci.h"

Error:

src/test_app/main.cpp:7:10: fatal error: soci.h: No such file or directory
 #include "soci.h"
          ^~~~~~~~
compilation terminated.

And it seems like the header files are further down, this will error differently:

#include "soci/soci.h"

Error:

build/_deps/soci-src/src/core/../../include/soci/soci-platform.h:24:10: fatal error: soci/soci-config.h: No such file or directory
 #include "soci/soci-config.h" // for SOCI_HAVE_CXX11
          ^~~~~~~~~~~~~~~~~~~~
compilation terminated.

tt4g commented 5 years ago

Try adding the soci include directory to the project include directory: include_directories — CMake 3.16.0-rc2 Documentation

example src/CMakeLists.txt:

cmake_minimum_required(VERSION 3.11)

set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)

include(cmake/spdlog.cmake)
include(cmake/soci.cmake)

+ include_directories("${PATH_TO_SOCI_INCLUDE_DIR}")

add_subdirectory(test_app)

msis commented 5 years ago

+ include_directories("${PATH_TO_SOCI_INCLUDE_DIR}")

But isn't the purpose of adding the dependency to avoid that?

PS: I'm trying to avoid such an include as it is not recommended anymore...

tt4g commented 5 years ago

I suggested include_directories() as the simplest solution, with priority on problem solving. Because I thought it would be difficult to provide a better solution for complex CMake projects.

Of course you can use target_include_directories() to minimize the impact (e.g. target_include_directories(test_app PUBLIC "${PATH_TO_SOCI_INCLUDE_DIR}")).

target_include_directories — CMake 3.16.0-rc2 Documentation

msis commented 5 years ago

I understand, but it didn't solve the issue neither. Here's what I out in the test_app/CMakeLists.txt:

set(PATH_TO_SOCI_INCLUDE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/../../build/_deps/soci-src/include)
message(STATUS ${PATH_TO_SOCI_INCLUDE_DIR})
target_include_directories(test_app
    PRIVATE
        ${PATH_TO_SOCI_INCLUDE_DIR}
)

And I still get the same error... So I suppose it was exposed right the first time..

tt4g commented 5 years ago

Is the following error message? https://github.com/SOCI/soci/issues/762#issuecomment-544324906:

build/_deps/soci-src/src/core/../../include/soci/soci-platform.h:24:10: fatal error: soci/soci-config.h: No such file or directory
 #include "soci/soci-config.h" // for SOCI_HAVE_CXX11
          ^~~~~~~~~~~~~~~~~~~~
compilation terminated.

If so, the file generated by the soci project's CMAKELists.txt may not exist.

https://github.com/SOCI/soci/blob/dbf93977c64bacdfc93a0bc46188c2167a3284f0/CMakeLists.txt#L152-L155

Does soci-config.h exist in ${PROJECT}/build/_deps/soci-src/include/soci? .

tt4g commented 5 years ago

@msis I created a small project to build soci. Please refer to this project: https://github.com/tt4g/soci_issue_762/tree/master

The problem of missing soci/soci-config.h is solved by CMake in-source build: https://github.com/tt4g/soci_issue_762/commit/68faa17ab53da859c878e4749a6c610fb2cfe3e2

msis commented 5 years ago

Does soci-config.h exist in ${PROJECT}/build/_deps/soci-src/include/soci? .

So that file isn't there. It's in ${PROJECT}/build/_deps/soci-build/include/soci

Is it possible the include files are not copied correctly? Because that is the only header file in that directory.

msis commented 5 years ago

@tt4g, I resolved the issue by changing the following in the soci/CMakeLists.txt:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 004dd07..2125bd1 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -149,7 +149,7 @@ set(INCLUDEDIR "include" CACHE PATH "The directory to install includes into.")
 ###############################################################################
 # Configuration files
 ###############################################################################
-set(CONFIG_INCLUDE_DIR ${CMAKE_CURRENT_BINARY_DIR}/include)
+set(CONFIG_INCLUDE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/include)
 install(DIRECTORY ${CONFIG_INCLUDE_DIR}/soci DESTINATION ${INCLUDEDIR})
 set(CONFIG_FILE_IN "include/soci/soci-config.h.in")
 set(CONFIG_FILE_OUT "${CONFIG_INCLUDE_DIR}/soci/soci-config.h")

I have this as a patch during the FetchContent_Declare() in soci.cmake:

cmake_minimum_required(VERSION 3.11)

message(STATUS "Extern: SOCI pre4.0+git_blksail")

set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)

include(FetchContent)

FetchContent_Declare( soci
    GIT_REPOSITORY https://github.com/SOCI/soci
    GIT_TAG master
    GIT_SHALLOW ON
    PATCH_COMMAND git apply "${CMAKE_SOURCE_DIR}/src/patches/soci.patch"
)
FetchContent_GetProperties(soci)
if(NOT soci_POPULATED)
    set(SOCI_STATIC ON)
    set(SOCI_SHARED ON)
    set(SOCI_TESTS OFF)
    # set(SOCI_ASAN OFF)
    set(SOCI_CXX11 ON)
    set(SOCI_LIBDIR lib)
    set(WITH_SQLITE3 ON)
    set(WITH_POSTGRESQL ON)
    set(WITH_BOOST OFF)
    set(WITH_DB2 OFF)
    set(WITH_ODBC OFF)
    set(WITH_ORACLE OFF)
    set(WITH_MYSQL OFF)
    set(SOCI_EMPTY OFF)
    FetchContent_Populate(soci)
    add_subdirectory(
        ${soci_SOURCE_DIR}
        ${soci_BINARY_DIR}
        EXCLUDE_FROM_ALL)
endif()

msis commented 5 years ago

While this solution is a hack, it let's users import soci as dependency.

I will leave it to the maintainers to close it if they see fit.

PS: before v4 , it would be great to give CMake some love. Some great resources:

tt4g commented 5 years ago

So that file isn't there. It's in ${PROJECT}/build/_deps/soci-build/include/soci

Is it possible the include files are not copied correctly? Because that is the only header file in that directory.

@msis Probably the difference between CMake in-souce builds and out-of-source builds. If the output destination (binary directory) when building is in-souce build, it will be the source directory. On the other hand, if it is out-of-source build, it becomes a dedicated directory.

Since SOCI outputs a header file at build time, I think that soci/soci-config.h could not be found if the source directory and another directory were specified as the binary directory.

Your project can now be built because you changed CONFIG_INCLUDE_DIR to the source directory.

PS: It's very difficult to build external third party projects together in your own project like this one. If possible, install the package on your system, or consider introducing a C/C ++ package manager.

msis commented 5 years ago

PS: It's very difficult to build external third party projects together in your own project like this one. If possible, install the package on your system, or consider introducing a C/C ++ package manager.

@tt4g Which one do you recommend? I looked at conan and vcpkg but CMake looked easier still...

tt4g commented 5 years ago

PS: It's very difficult to build external third party projects together in your own project like this one. If possible, install the package on your system, or consider introducing a C/C ++ package manager.

@tt4g Which one do you recommend? I looked at conan and vcpkg but CMake looked easier still...

@msis Sorry for letting you expect. I don't know any C/C++ package manager that fully supports soci (I found some published conan recipes but it wasn't satisfactory). As far as I know, the most stable way to use soci in a CMake project is to install it with CMake.

mloskot commented 5 years ago

@msis

PS: before v4 , it would be great to give CMake some love.

It would, but it is not going to happen, unless someone does the job (not much time left!). I am not able to pursue this task.

If anyone is interested in the task, I have started CMake modernisation here https://github.com/SOCI/soci/tree/wip/ml/modern-cmake

Some great resources:

I have been well aware of all those for quite some time, including this https://crascit.com/professional-cmake/ which is single best one.

msis commented 5 years ago

Thanks @mloskot .

What's the minimum CMake version you're ready to have?

mloskot commented 5 years ago

Personally, I'm always on the latest release. For SOCI, not sure what would be suitable, certainly not older than 3.5 or 3.8. @vadz may want to comment.

I'm not sure if it is feasible to get the CMake rewrite for SOCI 4.0.0 which I'm ready to start releasing any time soon (in fact, release candidate 1 could have happened even now).

craigscott-crascit commented 5 years ago

In case it's useful, here's a CMake function we use at work to bring in soci into our build (note the restrictions and assumptions as mentioned in the comments, etc.):

# Download soci using dependency details from "soci" and adds
# it to the build via add_subdirectory(). It will be assumed
# that Boost should be enabled unless the NO_BOOST option
# is given.
#
# Arguments to the function should be the required backends
# to enable in the soci build (all lowercase). Eg:
#
#  myCustomDependencyAddSoci(oracle postgresql)
#
function(myCustomDependencyAddSoci)
    if(NOT UNIX)
        message(FATAL_ERROR "Soci support only implemented for Unix")
    endif()

    include(FetchContent)
    FetchContent_GetProperties(soci)
    if(soci_POPULATED)
        return()
    endif()
    FetchContent_Populate(soci)

    # Set up for our customizations before bringing soci into the build
    set(options   "NO_BOOST")
    set(singleVal "")
    set(multiVal  "")
    cmake_parse_arguments(ARGS "${options}" "${singleVal}" "${multiVal}" ${ARGN})

    if(ARGS_NO_BOOST)
        set(WITH_BOOST OFF CACHE INTERNAL "")
    else()
        find_package(Boost QUIET)
        if(Boost_FOUND)
            set(WITH_BOOST ON  CACHE INTERNAL "")
        else()
            set(WITH_BOOST OFF CACHE INTERNAL "")
        endif()
    endif()

    set(requestedBackends ${ARGS_UNPARSED_ARGUMENTS})
    set(supportedBackends
        odbc
        oracle
        postgresql
        sqlite3
        db2
        firebird
        mysql
    )

    # Assumes you want C++11, no soci tests and build shared libs
    set(SOCI_CXX_C11 ON  CACHE INTERNAL "")
    set(SOCI_TESTS   OFF CACHE INTERNAL "")
    set(SOCI_STATIC  OFF CACHE INTERNAL "")

    foreach(backend IN LISTS supportedBackends)
        string(TOUPPER "${backend}" beUpper)
        string(TOLOWER "${backend}" beLower)
        if("${beLower}" IN_LIST requestedBackends)
            set(WITH_${beUpper} ON  CACHE INTERNAL "")
        else()
            set(WITH_${beUpper} OFF CACHE INTERNAL "")
        endif()
    endforeach()

    # Take over where soci will install things and under what component name
    include(GNUInstallDirs)
    set(BINDIR ${CMAKE_INSTALL_BINDIR} CACHE INTERNAL "")
    set(LIBDIR ${CMAKE_INSTALL_LIBDIR} CACHE INTERNAL "")
    set(CMAKE_INSTALL_DEFAULT_COMPONENT_NAME soci)

    # Because of manually defined SOCI_VERSION
    set(CMAKE_POLICY_DEFAULT_CMP0048 NEW)

    add_subdirectory(${soci_SOURCE_DIR} ${soci_BINARY_DIR})

    # Ensure we have namespaced targets. These are what projects
    # should reference directly, not the unnamespaced soci_* targets.
    foreach(lib IN LISTS requestedBackends ITEMS core empty)
        if(NOT TARGET SOCI::soci_${lib})
            add_library(SOCI::soci_${lib} ALIAS soci_${lib})
        endif()
    endforeach()

    # Fix missing information in some targets (haven't tested all backends)
    set_property(TARGET soci_core APPEND PROPERTY
        INTERFACE_INCLUDE_DIRECTORIES
            $<BUILD_INTERFACE:${soci_BINARY_DIR}/include>
    )
    if(WITH_BOOST)
        set_property(TARGET soci_core APPEND PROPERTY
            INTERFACE_INCLUDE_DIRECTORIES
                $<TARGET_PROPERTY:Boost::boost,INTERFACE_INCLUDE_DIRECTORIES>
        )
        set_property(TARGET soci_core APPEND PROPERTY
            INTERFACE_COMPILE_DEFINITIONS SOCI_USE_BOOST
        )
    endif()
    if("oracle" IN_LIST requestedBackends)
        set_property(TARGET soci_oracle APPEND PROPERTY
            INTERFACE_INCLUDE_DIRECTORIES $<BUILD_INTERFACE:${ORACLE_HOME}/sdk/include>
        )
    endif()

    # Work around build warning that gets turned into an error when building
    # for release with more recent compilers. The warning seems questionable
    # for the specific case that gets triggered in vector-into-type.cpp.
    if(WITH_ORACLE AND CMAKE_CXX_COMPILER_ID MATCHES "Clang|GNU")
        target_compile_options(soci_oracle PRIVATE -Wno-maybe-uninitialized)
    endif()

endfunction()

It assumes you've declared the details you want before calling the function. Use it something like the following:

FetchContent_Declare(soci
    GIT_REPOSITORY https://github.com/SOCI/soci
    GIT_TAG 125753ff2fa8457b980fc4186606b9b08b2612b5
)

myCustomDependencySoci(oracle postgresql)

SOCI / soci

`soci` not found when adding as subproject #762

Project description

`src/CMakeLists.txt`

`src/test_app/CMakeLists.txt`

`spdlog.cmake`

`soci.cmake`