search

ceres-solver / ceres-solver

A large scale non-linear optimization library
http://ceres-solver.org/
Other
3.73k stars 1.02k forks source link

FAILURE (Can't use SPARSE_NORMAL_CHOLESKY with SUITESPARSE because SuiteSparse was not enabled when Ceres was built.) #197

Closed edgarriba closed 8 years ago

edgarriba commented 8 years ago

Hi guys,

I'm using TheiaSfM which has ceres as a dependency and got the following exception error when running the global pipeline:

Termination: FAILURE (Can't use SPARSE_NORMAL_CHOLESKY with SUITESPARSE because SuiteSparse was not enabled when Ceres was built.)

I'm pretty sure that I compiled ceres with a sparse solver (tried with Suitesparse, CXSparse and Eigensparse). Did you notice any similar issue in this direction?

PD: you can check the short chat I had with Theia author **** and the complete log I got.

sweeneychris commented 8 years ago

@edgarriba It would be helpful if you could include the CMake output when you are building Ceres

edgarriba commented 8 years ago

sure. This is my CMake output:

-- Detected Ceres version: 1.12.0 from /home/eriba/software/ceres-solver/include/ceres/version.h -- Found Eigen version 3.2.0: /usr/include/eigen3

=============================================================== Enabling the use of Eigen as a sparse linear algebra library for solving the nonlinear least squares problems. Enabling this option results in an LGPL licensed version of Ceres Solver as the Simplicial Cholesky factorization in Eigen is licensed under the LGPL. WARNING:

Your version of Eigen is older than version 3.2.2. The performance of SPARSE_NORMAL_CHOLESKY and SPARSE_SCHUR linear solvers will suffer.

-- A library with BLAS API found. -- Found LAPACK library: /usr/lib/liblapack.so;/usr/lib/libf77blas.so;/usr/lib/libatlas.so -- Found BLAS library: /usr/lib/libf77blas.so;/usr/lib/libatlas.so -- A library with BLAS API found. -- Found AMD library: /usr/local/lib/libamd.so -- Found AMD header in: /usr/local/include -- Found CAMD library: /usr/local/lib/libcamd.so -- Found CAMD header in: /usr/local/include -- Found COLAMD library: /usr/local/lib/libcolamd.so -- Found COLAMD header in: /usr/local/include -- Found CCOLAMD library: /usr/local/lib/libccolamd.so -- Found CCOLAMD header in: /usr/local/include -- Found CHOLMOD library: /usr/local/lib/libcholmod.so -- Found CHOLMOD header in: /usr/local/include -- Found SuiteSparseQR library: /usr/local/lib/libspqr.so -- Found SuiteSparseQR header in: /usr/local/include -- Found Intel Thread Building Blocks (TBB) library: /usr/lib/libtbb.so, assuming SuiteSparseQR was compiled with TBB. -- Found Intel Thread Building Blocks (TBB) Malloc library: /usr/lib/libtbbmalloc.so -- Found SuiteSparse_config library: /usr/local/lib/libsuitesparseconfig.so -- Found SuiteSparse_config header in: /usr/local/include -- Adding librt: /usr/lib/x86_64-linux-gnu/librt.so to SuiteSparse_config libraries (required on Linux & Unix [not OSX] if SuiteSparse is compiled with timing). -- Found METIS library: /usr/local/lib/libmetis.so. -- Found SuiteSparse 4.5.0, building with SuiteSparse. -- Found CXSparse version: 3.1.7, building with CXSparse. -- No preference for use of exported gflags CMake configuration set, and no hints for include/library directories provided. Defaulting to preferring an installed/exported gflags CMake configuration if available. -- Found installed version of gflags: /home/eriba/software/OpenDroneMap-edgarriba/SuperBuild/build/gflags -- Detected broken gflags install in: /home/eriba/software/OpenDroneMap-edgarriba/SuperBuild/build/gflags, version: 2.1.2 <= 2.1.2 which defines gflags_LIBRARIES = gflags which is not an imported CMake target, see: https://github.com/gflags/gflags/issues/110. Attempting to fix by detecting correct gflags target. -- Found valid gflags target: gflags-static, updating gflags_LIBRARIES. -- Detected gflags version: 2.1.2 -- Found Google Flags header in: /home/eriba/software/OpenDroneMap-edgarriba/SuperBuild/build/gflags/include, in namespace: google -- Found Google Log header in: /usr/include -- Building with OpenMP. -- Looking for C++ include unordered_map -- Looking for C++ include unordered_map - not found -- Looking for C++ include tr1/unordered_map -- Looking for C++ include tr1/unordered_map - found -- Found tr1/unordered_map/set in std::tr1 namespace. -- Looking for C++ include memory -- Looking for C++ include memory - found -- Performing Test HAVE_SHARED_PTR_IN_STD_NAMESPACE -- Performing Test HAVE_SHARED_PTR_IN_STD_NAMESPACE - Failed -- Performing Test HAVE_SHARED_PTR_IN_TR1_NAMESPACE -- Performing Test HAVE_SHARED_PTR_IN_TR1_NAMESPACE - Failed -- Looking for C++ include tr1/memory -- Looking for C++ include tr1/memory - found -- Performing Test HAVE_SHARED_PTR_IN_TR1_NAMESPACE_FROM_TR1_MEMORY_HEADER -- Performing Test HAVE_SHARED_PTR_IN_TR1_NAMESPACE_FROM_TR1_MEMORY_HEADER - Success -- Found shared_ptr in std::tr1 namespace using <tr1/memory> header. -- Building Ceres as a static library. -- Enabling CERES_USE_EIGEN_SPARSE in Ceres config.h -- Enabling CERES_USE_OPENMP in Ceres config.h -- Enabling CERES_HAVE_PTHREAD in Ceres config.h -- Enabling CERES_HAVE_RWLOCK in Ceres config.h -- Enabling CERES_TR1_UNORDERED_MAP in Ceres config.h -- Enabling CERES_TR1_MEMORY_HEADER in Ceres config.h -- Enabling CERES_TR1_SHARED_PTR in Ceres config.h -- Do not build any example. -- Configuring done -- Generating done -- Build files have been written to: /home/eriba/software/ceres-solver/build

sweeneychris commented 8 years ago

"-- Enabling CERES_USE_EIGEN_SPARSE in Ceres config.h"

disable CERES_USE_EIGEN_SPARSE

sandwichmaker commented 8 years ago

why does that make a difference?

On Mon, Feb 29, 2016 at 12:13 PM Chris Sweeney notifications@github.com wrote:

"-- Enabling CERES_USE_EIGEN_SPARSE in Ceres config.h"

disable CERES_USE_EIGEN_SPARSE

— Reply to this email directly or view it on GitHub https://github.com/ceres-solver/ceres-solver/issues/197#issuecomment-190364482 .

sweeneychris commented 8 years ago

Not sure why CERES_NO_SUITESPARSE is being set unless it is being overwritten somewhere. I never touch this (or any other Ceres external libs params/macros) in Theia.

sandwichmaker commented 8 years ago

can you replicate this bug chris?

On Mon, Feb 29, 2016 at 12:26 PM Chris Sweeney notifications@github.com wrote:

Not sure why CERES_NO_SUITESPARSE is being set unless it is being overwritten somewhere. I never touch this (or any other Ceres libs params) in Theia.

— Reply to this email directly or view it on GitHub https://github.com/ceres-solver/ceres-solver/issues/197#issuecomment-190370493 .

sweeneychris commented 8 years ago

No I have not been able to

On Mon, Feb 29, 2016 at 12:27 PM Sameer Agarwal notifications@github.com wrote:

can you replicate this bug chris?

On Mon, Feb 29, 2016 at 12:26 PM Chris Sweeney notifications@github.com wrote:

Not sure why CERES_NO_SUITESPARSE is being set unless it is being overwritten somewhere. I never touch this (or any other Ceres libs params) in Theia.

— Reply to this email directly or view it on GitHub < https://github.com/ceres-solver/ceres-solver/issues/197#issuecomment-190370493

.

— Reply to this email directly or view it on GitHub https://github.com/ceres-solver/ceres-solver/issues/197#issuecomment-190370786 .

sandwichmaker commented 8 years ago

@alexsmac any idea how to debug this?

edgarriba commented 8 years ago

Does it has something to do the fact that ceres is built as static?

edgarriba commented 8 years ago

another thing that I've noticed is that during installation nothing is copied in /usr/local/share/Ceres

alexsmac commented 8 years ago

What does the first line from Solver::Summary::FullReport() say for your current Ceres installation? The one you posted earlier for Theia had this:

Solver Summary (v 1.10.0-lapack-no_openmp)

when it should be something more like this:

Solver Summary (v 1.12.0-eigen-(3.2.5)-lapack-suitesparse-(4.4.4)-cxsparse-(3.1.4)-no_openmp)

given the configuration output you posted above.

I suspect that Theia is linking/linked (if static) against an older version of Ceres installed on your machine. Checking the contents of Ceres_DIR in the advanced view in the CMake GUI when building Theia should tell you which one it is using. Note that this should be the path to the directory containing CeresConfig.cmake (e.g. /usr/local/lib/cmake/Ceres). To answer your question about /usr/local/share/Ceres, as of 1.12+, that is no longer used, we use the (potentially) architecture dependent install location: /usr/local/lib/cmake/Ceres.

One way that Theia could be picking up an old Ceres build would be if you exported the Ceres build directory (instead of installing it) by enabling EXPORT_BUILD_DIR. You can easily check for this by looking at the contents of ~/.cmake/packages/Ceres. If that contains any files (which should have SHA codes for names, and contain the full path to a Ceres build directory) which point to existing Ceres build directories, then they will be used in preference to an installed version of Ceres.

edgarriba commented 8 years ago

Definitely we have a Poltergeist here.

I've checked the Ceres_DIR from Theia and is set with its default value (/usr/local/lib/cmake/Ceres). I've also checked in ~/.cmake/packages/Ceres and you were right, there was a file, but it was pointing to the new ceres version. To be sure I removed this file and compiled again setting the EXPORT_BUILD_DIR flag to OFF.

Same result.

alexsmac commented 8 years ago

But what does the first line from Solver::Summary::FullReport() say for your current Ceres installation? Are you still seeing this:

Solver Summary (v 1.10.0-lapack-no_openmp)

? Or something more like this:

Solver Summary (v 1.12.0-eigen-(3.2.5)-lapack-suitesparse-(4.4.4)-cxsparse-(3.1.4)-no_openmp)

edgarriba commented 8 years ago

yes, same summary as before. Each time I uninstall ceres, theia complains about that, so theia is linking the current ceres version.

sweeneychris commented 8 years ago

Is it possible for Theia to detect at Cmake time if Ceres was built with SuiteSparse? If so, I could at least have Cmake halt since SuiteSparse is needed for sparse BA in Theia

alexsmac commented 8 years ago

@sweeneychris - try this: https://ceres-solver-review.googlesource.com/#/c/7370

It still needs a docs update, but should give you an idea of how it might work - feedback welcome :)

alexsmac commented 8 years ago

@sweeneychris the CL has now been merged into master, so you can do things like this:

find_package(Ceres REQUIRED SuiteSparse)

or:

find_package(Ceres REQUIRED SparseLinearAlgebraLibrary)

to get check for at least one sparse matrix backend) (SuiteSparse OR EigenSparse OR CXSparse).

This will be backwards compatible in a way, in that find_package(Ceres REQUIRED SparseLinearAlgebraLibrary) should report Ceres as found even on older Ceres installs that predate this change. However - it will obviously report Ceres as found without checking whether the specified modules are present.

sweeneychris commented 8 years ago

Awesome, this is great @alexsmac thanks for the patch!

@edgarriba If you sync to the latest master branch of Theia, it now explicitly requries SuiteSparse (independently of this Ceres patch) since CHOLMOD is being used for solving some large sparse systems.

soulslicer commented 6 years ago

I am still experiencing this issue in my project on gcc4.8. How exactly do you fix this?