dlib not using prebuild OpenBlas (on Mac)

[tobiasruf]

Intro

Use custom built OpenBlas static lib with dlib as external library in CMake project.

We're build dlib as external project in our CMake project and we would like to build OpenBlas as well as external project due to more flexibility for different platforms and clean build process.

Our current setup looks like that:

include(ExternalProject)

Set( DLIB_VERSION v19.16 )
Set( DLIB_VERSION_MSVC 19.16.0 )

if( DEFINED 3RD_PARTY_TOOLCHAIN_FILE )
   Set( TOOLCHAIN -DCMAKE_TOOLCHAIN_FILE=${3RD_PARTY_TOOLCHAIN_FILE} )
endif()

if( NOT CMAKE_CROSSCOMPILING )
   List( APPEND ADDITIONAL_FLAGS
         -DUSE_AVX_INSTRUCTIONS=ON
       )
endif()

set( DLIB_INSTALL ${CMAKE_CURRENT_BINARY_DIR}/install )

# Get opeblas lib location (imported target from external project)
get_target_property( OPENBLAS_LIB openblas IMPORTED_LOCATION )

# patch the find_blas from dlib to directly set the pass.
configure_file( find_blas.cmake.in find_blas.cmake @ONLY )

ExternalProject_Add(
   dlib_
   URL               https://github.com/davisking/dlib/archive/${DLIB_VERSION}.tar.gz
   CMAKE_ARGS        ${TOOLCHAIN}
                     -DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}
                     ${ADDITIONAL_FLAGS}
                     -DCMAKE_INSTALL_PREFIX=${DLIB_INSTALL}

   PATCH_COMMAND     ${CMAKE_COMMAND} -E copy ${CMAKE_CURRENT_BINARY_DIR}/find_blas.cmake
                                              ${CMAKE_CURRENT_BINARY_DIR}/dlib_-prefix/src/dlib_/dlib/cmake_utils/find_blas.cmake

   TEST_COMMAND      ""
)

add_dependencies( dlib_ openblas_ )

Set_Property( DIRECTORY ${CMAKE_SOURCE_DIR} APPEND 
              PROPERTY INCLUDE_DIRECTORIES ${DLIB_INSTALL}/include )

Add_Library( dlib STATIC IMPORTED GLOBAL )

add_dependencies( dlib dlib_ )

Set( LIB_PRE  ${CMAKE_STATIC_LIBRARY_PREFIX} )
Set( LIB_POST ${CMAKE_STATIC_LIBRARY_SUFFIX} )

#if ( CMAKE_SYSTEM_NAME STREQUAL "Darwin" )
#   Set( DLIB_LINK_INTERFACE_LIBS "cblas;clapack" )
#endif ()

Set( DLIB_LINK_INTERFACE_LIBS openblas )

if ( NOT MSVC )
   Set_Target_Properties( dlib PROPERTIES
                          IMPORTED_LOCATION ${DLIB_INSTALL}/lib/${LIB_PRE}dlib${LIB_POST} 
                        )
else()
   Set( numbits 64 )
   set_target_properties( dlib PROPERTIES 
                               IMPORTED_LOCATION_DEBUG "${DLIB_INSTALL}/lib/dlib${DLIB_VERSION_MSVC}_debug_${numbits}bit_msvc${MSVC_VERSION}${LIB_POST}"
                               IMPORTED_LOCATION_RELEASE "${DLIB_INSTALL}/lib/dlib${DLIB_VERSION_MSVC}_release_${numbits}bit_msvc${MSVC_VERSION}${LIB_POST}"
                               IMPORTED_LOCATION_MINSIZEREL "${DLIB_INSTALL}/lib/dlib${DLIB_VERSION_MSVC}_minsizerel_${numbits}bit_msvc${MSVC_VERSION}${LIB_POST}"
                               IMPORTED_LOCATION_RELWITHDEBINFO "${DLIB_INSTALL}/lib/dlib${DLIB_VERSION_MSVC}_relwithdebinfo_${numbits}bit_msvc${MSVC_VERSION}${LIB_POST}"
                        )
endif()

Set_Target_Properties( dlib PROPERTIES
                       IMPORTED_LINK_INTERFACE_LIBRARIES "${DLIB_LINK_INTERFACE_LIBS}" )

find_blas.cmake.in

SET(blas_found TRUE )
SET(lapack_found TRUE )

Set( blas_libraries @OPENBLAS_LIB@ )

SET(found_intel_mkl 0)
SET(found_intel_mkl_headers 0)
SET(lapack_with_underscore 0)
SET(lapack_without_underscore 0)

Expected Behavior

The final build should use the openblas symbols from libopenblas.a.

Current Behavior

The final build does not use the symbols. If I remove the openblas from the LINK_INTERFACE_LIBRARIES I don't get a link error.

NOTE: If I use the standard way with find_blas.cmak the dlib CMake project finds the Mac OS cblas and clapack, but the symbols don't get used, too. I see this when using DNNs which are very slow. In another test project it worked with 19.15., but I wasn't able to reproduce this again for the new project.

Under Ubuntu the above approach seems to work, since I get undefined symbols if remove openblas.

Any ideas what could be the problem?

• Version: 19.16.0
• Where did you get dlib: Download source code from Github relase
• Platform: Mac OS
• Compiler: clang

[tobiasruf]

One additional suggestion: Would it be an idea to enable the user to "inject" the blas lib with CMake parameters more easily?

We follow the philosophy to build our project incl. 3rd-party libs wit the same compiler version.

[davisking]

My experience with ExternalProject_Add is that it's basically broken to the point of being useless. In particular, it doesn't seem to integrate well, or at all really, with cmake's built in dependency management.

[tobiasruf]

In my experience ExternalProject_Add works well, since it lets you do the common steps (cmake, make, make install) directly from your project with your compiler (important for cross compiling). However, that's another discussion. ;-)

I will try to "inject" my custom OpenBlas library with the standard CMake dlib setup without ExternalProject_Add and report my experience here.

[tobiasruf]

No problem with Dlib and CMake at all. Had some linker issue with non-optimized BLAS symbols coming from another static lib, that we use.

Using my own Openblas build, like shown above, works good.