search

spack / spack

A flexible package manager that supports multiple versions, configurations, platforms, and compilers.
https://spack.io
Other
4.21k stars 2.24k forks source link

Using spack packages in CMake projects #20566

Open JeffR1992 opened 3 years ago

JeffR1992 commented 3 years ago

I've installed OpenCV with spack using spack install opencv, and am now trying to use this spack version of OpenCV in a CMake project.

In my current CMake project, I'm using OpenCV version 3.2, that comes with Ubuntu 18.04, as follows:

find_package(OpenCV REQUIRED)
include_directories(${OpenCV_INCLUDE_DIRS})
target_link_libraries(main ${OpenCV_LIBS})

However, I'm a little uncertain as to how I can instead use the newer spack version of OpenCV in my CMake project. One attempt I tried was to find out where spack installed OpenCV, and then use this hard coded path to include header files and gain access to the OpenCV shared libraries.

Spack installed OpenCV into

~/spack/opt/spack/linux-ubuntu18.04-skylake/gcc-10.1.0/opencv-4.5.0-bgg64urfqe7473kwszrujfmqzrn3d7xw/

with bin, include, lib and share directories contained within this directory. As such, I tried using the spack version of OpenCV in my CMake project as follows:

include_directories(~/spack/opt/spack/linux-ubuntu18.04-skylake/gcc-10.1.0/opencv-4.5.0-
bgg64urfqe7473kwszrujfmqzrn3d7xw/include)
link_directories(~/spack/opt/spack/linux-ubuntu18.04-skylake/gcc-10.1.0/opencv-4.5.0-bgg64urfqe7473kwszrujfmqzrn3d7xw/lib)
target_link_libraries(main opencv_core)

However, when compiling my code I receive linking errors, and as such I was wondering if there's a cleaner way of using spack packages in CMake projects. Any help would be appreciated.

adamjstewart commented 3 years ago

The find_package(...) command in CMake should handle searching for and linking to OpenCV for you. You just have to tell CMake where to find OpenCV. If your project doesn't have flags to explicitly define where OpenCV is installed, setting CMAKE_PREFIX_PATH should work. I believe this is done automatically when you run spack load opencv or module load <opencv-module-name>.

JeffR1992 commented 3 years ago

Thanks @adamjstewart , unfortunately trying spack load opencv and then building with CMake, gives the following message: Found OpenCV: /usr (found version "3.2.0") So it looks like the default Ubuntu system version of OpenCV is still being found instead of the newer spack version.

I did have better luck with your suggestion of using CMAKE_PREFIX_PATH as follows: set(CMAKE_PREFIX_PATH "${CMAKE_PREFIX_PATH};~/spack/opt/spack/linux-ubuntu18.04-skylake/gcc-10.1.0/opencv-4.5.0-bgg64urfqe7473kwszrujfmqzrn3d7xw/lib/cmake/opencv4") which, when building, gives me: Found OpenCV: /home/alex/spack/opt/spack/linux-ubuntu18.04-skylake/gcc-10.1.0/opencv-4.5.0-bgg64urfqe7473kwszrujfmqzrn3d7xw (found version "4.5.0") However, this method seems a bit ugly, and instead of getting linking errors I now get source code errors that were not present when using OpenCV 3.2.0, for example:

error: ‘distanceTransform’ is not a member of ‘cv’
   99 |     cv::distanceTransform(inverse_occupancy_grid_map, distance_transform_map, cv::DIST_L2, cv::DIST_MASK_PRECISE, CV_32F);

But I'm wondering if this is due to API/header changes that were made to OpenCV between versions 3.2.0 and 4.5.0, so will have to dig a little deeper.

adamjstewart commented 3 years ago

Hmm, it seems to be present in the latest release. Did you add #include <opencv2/imgproc.hpp>?

JeffR1992 commented 3 years ago

Sorry yes you're right, I had commented out #include <opencv2/imgproc.hpp> in my code during testing. However, when I un-comment this include statement I unfortunately get a new set of errors which point to imgproc.hpp contained in /usr/include. Here's an example of the first error I get:

/usr/include/opencv2/imgproc.hpp:4124:56: error: ‘LINE_8’ was not declared in this scope; did you mean ‘__LINE__’?

But, if I print out the value of OpenCV_INCLUDE_DIRS I get: /home/alex/spack/opt/spack/linux-ubuntu18.04-skylake/gcc-10.1.0/opencv-4.5.0-bgg64urfqe7473kwszrujfmqzrn3d7xw/include/opencv4.

Furthermore, doing a search for imgproc.hpp in the spack folder doesn't return anything which is quite surprising, as I thought that this header would need to be present in spack in order to get the spack version of OpenCV working.

I case this additional piece of information may also be useful, printing out the OpenCV_LIBS variable gives me: opencv_core.

adamjstewart commented 3 years ago

Ah, if you need the imgproc module, you need to build opencv+imgproc. Same for any other modules you need.

JeffR1992 commented 3 years ago

Aha great, in that case is there a way to have spack download and build a "full" version of OpenCV? If I print out the OpenCV_LIBS variable for OpenCV version 3.2.0 that is located in the /usr directory I get quite a lot of libraries:

OpenCV_LIBS: opencv_calib3d;opencv_core;opencv_features2d;opencv_flann;opencv_highgui;opencv_imgcodecs;opencv_imgproc;opencv_ml;opencv_objdetect;opencv_photo;opencv_shape;opencv_stitching;opencv_superres;opencv_video;opencv_videoio;opencv_videostab;opencv_viz;opencv_aruco;opencv_bgsegm;opencv_bioinspired;opencv_ccalib;opencv_datasets;opencv_dpm;opencv_face;opencv_freetype;opencv_fuzzy;opencv_hdf;opencv_line_descriptor;opencv_optflow;opencv_phase_unwrapping;opencv_plot;opencv_reg;opencv_rgbd;opencv_saliency;opencv_stereo;opencv_structured_light;opencv_surface_matching;opencv_text;opencv_ximgproc;opencv_xobjdetect;opencv_xphoto

And it would be good to have the same libraries available when I download and build the updated OpenCV version 4.5.0 with spack.

adamjstewart commented 3 years ago

You can enable or disable whatever you want:

$ spack info opencv
...
Variants:
    Name [Default]                 Allowed values          Description
    ===========================    ====================    =====================================================

    build_type [RelWithDebInfo]    Debug, Release,         CMake build type
                                   RelWithDebInfo,         
                                   MinSizeRel              
    calib3d [off]                  on, off                 calib3d module
    contrib [off]                  on, off                 Adds in code from opencv_contrib.
    core [on]                      on, off                 Include opencv_core module into the OpenCV build
    cuda [off]                     on, off                 Build with CUDA
    cuda_arch [none]               none, 62, 80, 32,       CUDA architecture
                                   60, 70, 30, 61, 35,     
                                   50, 52, 13, 20, 21,     
                                   37, 53, 10, 12, 75,     
                                   86, 72, 11              
    cudacodec [off]                on, off                 Enable video encoding/decoding with CUDA
    dnn [off]                      on, off                 Build DNN support
    eigen [off]                    on, off                 Activates support for eigen
    fast-math [off]                on, off                 Enable -ffast-math (not recommended for GCC 4.6.x)
    features2d [off]               on, off                 features2d module
    flann [off]                    on, off                 flann module
    gtk [off]                      on, off                 Activates support for GTK
    highgui [off]                  on, off                 Include opencv_highgui module into the OpenCV build
    imgcodecs [off]                on, off                 Include opencv_imgcodecs module into the OpenCV build
    imgproc [off]                  on, off                 Include opencv_imgproc module into the OpenCV build
    ipo [off]                      on, off                 CMake interprocedural optimization
    ipp [off]                      on, off                 Activates support for IPP
    ipp_iw [off]                   on, off                 Build IPP IW from source
    jasper [off]                   on, off                 Activates support for JasPer
    java [off]                     on, off                 Activates support for Java
    jpeg [off]                     on, off                 Include JPEG support
    lapack [off]                   on, off                 Include Lapack library support
    ml [off]                       on, off                 Build ML support
    opencl [off]                   on, off                 Include OpenCL Runtime support
    opencl_svm [off]               on, off                 Include OpenCL Shared Virtual Memory support
    openclamdblas [off]            on, off                 Include OpenCL AMD OpenCL BLAS library support
    openclamdfft [off]             on, off                 Include OpenCL AMD OpenCL FFT library support
    openmp [off]                   on, off                 Activates support for OpenMP threads
    png [off]                      on, off                 Include PNG support
    powerpc [off]                  on, off                 Enable PowerPC for GCC
    pthreads_pf [off]              on, off                 Use pthreads-based parallel_for
    python [off]                   on, off                 Enables the build of Python extensions
    qt [off]                       on, off                 Activates support for QT
    shared [on]                    on, off                 Enables the build of shared libraries
    stitching [off]                on, off                 stitching module
    superres [off]                 on, off                 superres module
    tiff [off]                     on, off                 Include TIFF support
    ts [off]                       on, off                 Include opencv_ts module into the OpenCV build
    video [off]                    on, off                 video module
    videoio [off]                  on, off                 videoio module
    videostab [off]                on, off                 videostab module
    vsx [off]                      on, off                 Enable POWER8 and above VSX (64-bit little-endian)
    vtk [off]                      on, off                 Activates support for VTK
    zlib [off]                     on, off                 Build zlib from source

There isn't currently an "enable all variants" feature, but you can set the defaults to whatever you want in your packages.yaml.

JeffR1992 commented 3 years ago

Perfect, thanks @adamjstewart , so if I want to build additional packages on top of default opencv I should use the opencv+<package_name> syntax you mentioned above?

adamjstewart commented 3 years ago

Yes. In Spack we call things like opencv "packages" and things like +imgproc~openmp "variants". So if you build opencv+imgproc~openmp, it will build opencv with the opencv_imgproc module but without OpenMP support.