Latest HEAD does not skip HSL tests

[pat-s]

We were using latest HEAD to build ipopt recently and were facing build failures during the last days using the following setup

./coinbrew build Ipopt --prefix=/opt/ipopt --test --no-prompt --verbosity=2 -j=4 --reconfigure --without-hsl
./coinbrew install Ipopt --no-prompt --prefix=/opt/ipopt

During tests, the HSL tests do not seem to get skipped anymore even though HSL was skipped before during the build stage.

 ---- 8< ---- Start of test program output ---- 8< ----

*** Solve for 0 variables, feasible constraint, feasible bounds
Exception of type: DYNAMIC_LIBRARY_FAILURE in file "//Ipopt/src/Common/IpLibraryLoader.cpp" at line 67:
 Exception message: libhsl.so: cannot open shared object file: No such file or directory

Reverting to tag v1.0 solved the issue and let the build succeed again.

[tkralphs]

I think what is happening is that the new coinbrew is parsing config.yml, which has dependencies listed as "required" or "optional". This is to allow for automatically skipping the builds of dependencies that most users don't need. Right now, MUMPS is listed as optional, which means that if you also skip HSL, there is no linear solver available. I suppose this causes Ipopt just to try to use HSL anyway, but maybe @svigerske can comment?

In any case, the difference between the two versions is just that MUMPS doesn't currently get built. For now, if you add --enable-optional, everything should work with one caveat. It seems that if you built once without MUMPS and then you try to go back and build with MUMPS, but only reconfiguring and not building from scratch, it will still fail. You should just build from scratch.

In the long run, the solution is probably just to make MUMPS a required dependency.

[tkralphs]

And just to comment further with some hints.

• You don't need --no-prompt anymore, since you won't be prompted unless it's really necessary.
• You also don't need the install step anymore, this is done as part of the build.
• There are more fine-grained options for testing with --tests which takes values none, main, or all. --test is equivalent to --tests main.

[svigerske]

As @tkralphs said, the problem is that Ipopt is build without any linear solver. In that case, it tries to load HSL from a library at runtime. That fails here, so the test of Ipopt fails. There are no "HSL tests".

I made Mumps optional, because it is optional. That is, one can build Ipopt without Mumps as long as one provides another linear solver. (And often, other linear solvers give better performance, too.) I didn't understand "optional" as "dependencies that most users don't need". I had thought that coinbrew would still try to build optional dependencies, but just ignore them if their build fails, not ignore them from the beginning on. Well, so since that doesn't seem the case, I'll just mark all dependencies of Ipopt that coinbrew could build as "required".

[pat-s]

I can confirm HEAD is now working again 👍

I also made the suggested adjustments to our script.

FYI: I used the build instructions from the ipopt install manual - seems like they should also be updated then.

Thanks for the quick replies and fixes!

[tkralphs]

I made Mumps optional, because it is optional.

There is actually a third category, which is probably the right one for MUMPS: recommended. It is true that MUMPS is not required, but (as I guess you understood already) the problem is that coinbrew only allows each project individually to be optional, recommended, or required. What would be needed in this case is the ability to express that each linear solver individually is optional, but there is a requirement that there should be at least one. I could implement a scheme for expressing more complex requirements, I suppose, but it would make the whole thing much more complicated.

The "recommended" option is a middle ground that indicates "you probably want this in most cases, though it is not strictly required." Recommended packages are built by default, but you can add --disable-recommended to build only the truly required ones. Optional packages are not built by default, but you can enable building them with --enable-optional. If this is not documented anywhere, I will add it.

The scheme was just meant to avoid making every user build all the dependencies when some of them are clearly not needed to have a working code and/or are of no benefit in the most common use cases. I'm open to suggestions on this.

[svigerske]

recommended sounds good (well, not that Mumps is recommended as linear solver for Ipopt, but fine).

Implementing a logic to say "mumps, or hsl, or pardiso, or wsmp, or spral, or lapack-if-it-is-mkl" doesn't seem trivial enough that it would be worthwhile to have.