Configuring in Example/ on Linux (Ubuntu) for a subset of packages installed, searches cmake files for remaining uninstalled packages

[DimaLPorNLP]

The BUG

When I try to cmake in Example/build directory after successfully installed a subset of SuiteSparse packages (due to failure of SPEX to find gmp and mpfr libraries although they exists)

$ find /usr/ -name "libmpfr*"
/usr/lib/x86_64-linux-gnu/libmpfr.so.6
/usr/lib/x86_64-linux-gnu/libmpfr.so.6.1.0
/usr/share/doc/libmpfr6
$ find /usr/ -name "libgmp*"
/usr/lib/x86_64-linux-gnu/libgmp.so.10
/usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1
/usr/share/doc/libgmp10

I get the following problem which looks like a bug:

~/SuiteSparse/Example/build$ cmake -DCMAKE_PREFIX_PATH=`pwd`/../../build/lib/cmake ..

CMake Error at CMakeLists.txt:103 (find_package):
  By not providing "FindSuiteSparse_Mongoose.cmake" in CMAKE_MODULE_PATH this
  project has asked CMake to find a package configuration file provided by
  "SuiteSparse_Mongoose", but CMake did not find one.

  Could not find a package configuration file provided by
  "SuiteSparse_Mongoose" (requested version 3.3.4) with any of the following
  names:

    SuiteSparse_MongooseConfig.cmake
    suitesparse_mongoose-config.cmake

  Add the installation prefix of "SuiteSparse_Mongoose" to CMAKE_PREFIX_PATH
  or set "SuiteSparse_Mongoose_DIR" to a directory containing one of the
  above files.  If "SuiteSparse_Mongoose" provides a separate development
  package or SDK, be sure it has been installed.

Mongoose was not in the enabled packages. It seems that it will keep complaining for the next uninstalled package.

To Reproduce

I managed to configure and install a subset of the packages by commenting out the CMakeLists.txt line specifying SUITESPARSE_ENABLE_PROJECTS "all" and adding explicitly the packages to be enabled:

#set ( SUITESPARSE_ENABLE_PROJECTS "all" CACHE STRING
#    "Semicolon-separated list of SuiteSparse projects to be built (${SUITESPARSE_KNOWN_PROJ
ECTS}, or \"all\")" )

set(SUITESPARSE_ENABLE_PROJECTS "suitesparse_config;amd;btf;camd;ccolamd;colamd;cholmod;cxsp
arse;ldl;klu;umfpack" CACHE STRING "Enable specific SuiteSparse projects")

Then change directory to Example/build and try to cmake with:

~/SuiteSparse/Example/build$ cmake -DCMAKE_PREFIX_PATH=`pwd`/../../build/lib/cmake ..

Expected behavior I would expect that the example would understand what packages are installed and enable "example" code only for the packages installed. Is this the case?

Feedback on the new CMAKE system I find CMAKE extremely harder and more complicated to use than the simple Makefiles SuiteSparse was originally using. With simple Makefiles one knows what to expect and he knows how to fix a problem. With CMAKE the situation complicates exponentially.

[DrTimothyAldenDavis]

This is by design. The Example is not a primary library of SuiteSparse, just an example of how to build an application that depends on Suitesparse. It is meant to be a simple. It is meant to be revised to use just the libraries you want. So if you don't want to use a particular package in suitesparse then you just revise the Example to only use what you need

If I added conditional compilation to the Example, it would complicate it and obscure it.

[DrTimothyAldenDavis]

Regarding the Makefiles. They don't work in Windows, and they are hard to revise for all cases where Suitesparse is used.

The CI of suitesparse tests the libraries in many platforms, far more than what a Makefile can do.

[DimaLPorNLP]

Regarding the Makefiles. They don't work in Windows, and they are hard to revise for all cases where Suitesparse is used.

The CI of suitesparse tests the libraries in many platforms, far more than what a Makefile can do.

In Windows I never tried anything other than MSYS2. Do you imply that SuiteSparse default configuration supports only or works better with Visual Studio cmake integration and/or Visual Studio cl compiler?

[DimaLPorNLP]

This is by design. The Example is not a primary library of SuiteSparse, just an example of how to build an application that depends on Suitesparse. It is meant to be a simple. It is meant to be revised to use just the libraries you want. So if you don't want to use a particular package in suitesparse then you just revise the Example to only use what you need

If I added conditional compilation to the Example, it would complicate it and obscure it.

I thought so, but I was not 100% sure. The assumption was that the new cmake-design with all that enabling/disabling individual packages must have been applied to the whole design of SuiteSparse down to examples, so I was wondering whether or not I understood something wrong.

[DrTimothyAldenDavis]

On windows the cmake build system supports many things: msys2, MSVC,the MS compiler, the Intel compiler, and the list goes on. A makefile can't do that.

[DrTimothyAldenDavis]

Regarding the Example: yes, I could configure it to use just the packages that get compiled. But that wouldn't reflect an actual application that needs the libraries it needs and can't build without them. So it would just obscure the purpose of the Example.

[DimaLPorNLP]

On windows the cmake build system supports many things: msys2, MSVC,the MS compiler, the Intel compiler, and the list goes on. A makefile can't do that.

Thanks for the info. So this means we could create a VS project and compile SuiteSparse directly in MS Visual Studio? I understand this is a nice step forward from the Makefile system, but now there are also other alternatives which look way more simple to use and less chaotic than cmake. Are you aware of meson build system or the xmake build system. I used xmake for my own project and it took me a few minutes to setup it is doing everything almost automatically. Maybe one of these two could simplify a lot SuiteSparse configuration because it looks as simple (this is particularly true for xmake) as one would describe the project with the minimal amount of pseudocode, and the commands are very intuitive.

[DimaLPorNLP]

Regarding the Example: yes, I could configure it to use just the packages that get compiled. But that wouldn't reflect an actual application that needs the libraries it needs and can't build without them. So it would just obscure the purpose of the Example.

I see! Thanks for the info. In any case for people inexperienced with the chaos cmake brings, it would be very helpful and/or save new users a lot of time to have a few commented lines in CMakeLists.txt, as in the old makefile-based SuiteSparse, explaining how to enable or disable a subset of packages if "all" does not work for them or enabling/disabling additional features. It is not obvious at all. Even writing the package names in uppercase letters (as suggested by CoPilot) is not accepted by cmake (it cannot find them). Both on MSYS2 and Linux, SPEX fails for me although shared versions of libmpfr and libgmp are installed, but cmake cannot find them.

[mmuetzel]

explaining how to enable or disable a subset of packages if "all" does not work for them or enabling/disabling additional features.

The README file documents that already. If you think that something is missing or confusing in that file, that could be improved. But repeating the README across some CMake files is probably not the best way of documentation imho.

Both on MSYS2 and Linux, SPEX fails for me although shared versions of libmpfr and libgmp are installed, but cmake cannot find them.

The CI (on MSYS2 and Ubuntu), builds it just fine without issues. So, it must be something specific to your build environment or toolchain. https://github.com/DrTimothyAldenDavis/SuiteSparse/actions/workflows/build.yaml

[DimaLPorNLP]

The README file documents that already. If you think that something is missing or confusing in that file, that could be improved. But repeating the README across some CMake files is probably not the best way of documentation imho.

Probably it would be better to split the Quickstart section of the readme.md to a separate QuickStart file. Even though I found the Quickstart section when you told me, searching 4 times until I realize I should search for uppercase ENABLE instead of lowercase enable using vim, I was still not able to find the Quickstart section scrolling through the file several times. It is better such instructions to be separated for impatient users and contained at one place.

The CI (on MSYS2 and Ubuntu), builds it just fine without issues. So, it must be something specific to your build environment or toolchain. https://github.com/DrTimothyAldenDavis/SuiteSparse/actions/workflows/build.yaml Check the following. By installing libmpfr and libgmp which I never installed myself, I just installed the gcc compilers, the following libraries get installed

$ find /usr/ -name "libmpfr*"
/usr/lib/x86_64-linux-gnu/libmpfr.so.6
/usr/lib/x86_64-linux-gnu/libmpfr.so.6.1.0
/usr/share/doc/libmpfr6
$ find /usr/ -name "libgmp*"
/usr/lib/x86_64-linux-gnu/libgmp.so.10
/usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1
/usr/share/doc/libgmp10

These libraries cannot be found by cmake because it searches for .so files. Once I installed additional I do remember what packages, it continued asking for the static versions. How in the world am I supposed to know that it searches for static versions of these specific libraries when the libraries are installed from my package manager? For this purpose, a few instructions in the Quickstart regarding prerequisites would help a lot.

[mmuetzel]

In the CI, the soversion-suffixed libraries are detected without issues: E.g.: https://github.com/DrTimothyAldenDavis/SuiteSparse/actions/runs/11539239692/job/32118680669#step:9:3863

  -- major from gmp.h: #define __GNU_MP_VERSION            6
  -- minor from gmp.h: #define __GNU_MP_VERSION_MINOR      2
  -- patch from gmp.h: #define __GNU_MP_VERSION_PATCHLEVEL 1
  -- Found GMP: /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 (found suitable version "6.2.1", minimum required is "6.1.2")
  -- gmp version: 6.2.1
  -- gmp include: /usr/include/x86_64-linux-gnu
  -- gmp library: /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1
  -- gmp static:  /usr/lib/x86_64-linux-gnu/libgmp.a
  -- from mpfr.h file: #define MPFR_VERSION_STRING "4.1.0"
  -- Found MPFR: /usr/lib/x86_64-linux-gnu/libmpfr.so.6.1.0 (found suitable version "4.1.0", minimum required is "4.0.2")
  -- mpfr version: 4.1.0
  -- mpfr include: /usr/include
  -- mpfr library: /usr/lib/x86_64-linux-gnu/libmpfr.so.6.1.0
  -- mpfr static:  /usr/lib/x86_64-linux-gnu/libmpfr.a

If that isn't the case for you, something might be wrong with your build tools.

[mmuetzel]

Maybe, you didn't install the corresponding -dev packages that are needed for the headers in some distributions?

[DimaLPorNLP]

Maybe, you didn't install the corresponding -dev packages that are needed for the headers in some distributions?

No I did not install the -dev packages, I just installed the gcc and g++ and gfortran compilers. But if the -dev packages are needed shouldn't the wisdom of CMAKE inform somehow about that. In any case, under Linux, the .so.?.?.? should suffice for the linking. Why this does not happen it is up to CMAKE developers to tell us.

[mmuetzel]

In any case, under Linux, the .so.?.?.? should suffice for the linking.

Yes, they should suffice for linking (and they do). But you still need the headers for compiling.

But if the -dev packages are needed shouldn't the wisdom of CMAKE inform somehow about that.

CMake cannot (and should not) carry a database for which files are installed with which package for any existing distribution. And I don't think, there is any build system out there that does that. After all, it is up to the user to chose where they get the necessary dependencies from...

[DimaLPorNLP]

CMake cannot (and should not) carry a database for which files are installed with which package for any existing distribution. And I don't think, there is any build system out there that does that. After all, it is up to the user to chose where they get the necessary dependencies from...

But it knows where it searches for packages, libraries, etc depending on the system. So although what you wrote makes sense, it does not make sense not to inform you where it is searching and for what exactly is it searching and where it expects it to be found. So at list it should spit out a message such as:

Shared libraries for GMP were found in /path_of_gmp_shared, but headers such as gmp.h were not
found in /usr/include/gmp. Install the dev packages for your distribution for gmp,  and/or define the 
path where headers are stored using the variable: GMP_HEADERS if the location is not a standard one 
such as /usr/include/gmp, /usr/local/include/gmp.

Something like that is very informative and the user knows what to do next even if he is not familiar with cmake.