search

coin-or / Couenne

Convex Over and Under Envelopes for Nonlinear Estimation
Eclipse Public License 1.0
69 stars 7 forks source link

Errors when compiling and linking on Mac #44

Closed mlangiu closed 4 years ago

mlangiu commented 4 years ago

I fail to install Couenne on MAC OS 10.12.6 (16G29)

I obtained the source code with

svn co https://projects.coin-or.org/svn/Couenne/stable/0.5 Couenne

ran the ./get.XYZ scripts in all subdirectories of ThirdParty except for HSL, SCIP and SoPlex, for those I manually placed the coinhsl, scip and soplex folders obtained manually.

In Couenne/build I ran ../configure and then make but ran into the following issue:

Making all in ThirdParty/SoPlex
/Applications/Xcode.app/Contents/Developer/usr/bin/make  all-am
make[2]: *** No rule to make target `soplex/src/changesoplex.cpp', needed by `changesoplex.lo'.  Stop.
make[1]: *** [all] Error 2
make: *** [all-recursive] Error 1

Since I don't care too much about SoPlex, I gave up using it after a few attempts of using different settings and thus issued ../configure --without-soplex then make clean and make.

This way no errors appear to come up during compilation, but when the linker starts its work I get the error:

ld: warning: directory not found for option '-L/Users/admin/Couenne/build/lib'
ld: library not found for -lcoinasl
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [libClp.la] Error 1
make[2]: *** [all] Error 2
make[1]: *** [all-recursive] Error 1
make: *** [all-recursive] Error 1

The lib dir in my build folder is indeed missing, so it seems the preceeding steps didn't actually complete although I noticed no errors.

Compilers I am using:

I read in an old mailing list that the gfortran compilers by default compiles for 32 bit systems even if the host is 64, so I tried again, adding the same FFLAGS as in that thread, namely FFLAGS="-m64 -O3 -fomit-frame-pointer -pipe" but to no avail...

Here is the last config.log and the result of a subsequent make > make.log 2>&1: config.log make.log

tkralphs commented 4 years ago

Thanks for reporting this. Indeed, I don't believe Couenne can use SCIP or Soplex as subsolvers, so that is probably moot, although it would be helpful if you would report this failure to https://github.com/coin-or-tools/ThirdParty-Soplex. The missing -lcoinasl is because the ASL library is apparently not where it was expected to be.

You are doing a monolithic build, so nothing is installed until the end, which is why you did not see the lib directory in your build directory---this part is normal. However, there should be a libcoinasl.0.0.0.dylib in ThirdParty/ASL/.libs. Your output from the make of ASL is completely different than mine, so I have the feeling that you must have modified something. ASL usually builds with one single compile command that builds all source files rather than building them one by one, as your output shows. Most importantly, you are missing some lines at the end that indicate the library is being built. 

clang -dynamiclib -single_module  -o .libs/libcoinasl.0.0.0.dylib ...

Did you modify the get.ASL script or download a different version yourself or anything like that?

Just to mention, we have recently changed the recommended build procedure to one that is compatible with git in anticipation of moving all source code management over to Github. YOu might want to try building with coinbrew, as documented here. We are also working on homebrew recipes, which we have now up until Cbc (https://github.com/coin-or-tools/homebrew-coinor).

svigerske commented 4 years ago

Looks like ThirdParty/ASL was build using a more recent version of ThirdParty/ASL than the one supported by Counene. Couenne (stable/0.5) specifies in its svn:externals and Dependencies branch stable/1.4 for ASL for a reason. In the log, however, it seems that trunk or stable/2.0 was used, which might not work with current Couenne.

Regarding SoPlex, it might be that a SoPlex version different than the one excepted by ThirdParty/SoPlex was used. The INSTALL file in ThirdParty/SoPlex specifies which version it expects.

Couenne may use SCIP for its feasibility pump heuristic (I'm not sure whether that it enabled by default, though) and may use SoPlex as alternative to CLP as LP Solver (though underlying Cbc may prefer to use Clp).

tkralphs commented 4 years ago

Yes, I thought it looked like the ASL used was employing the newer build tools version. That would explain it.

mlangiu commented 4 years ago

Thank you for your quick replies Stefan and Ted!

Let me address your comments one by one:

You are doing a monolithic build, so nothing is installed until the end, which is why you did not see the lib directory in your build directory---this part is normal

This seems very strange to me, all previous projects with dependencies that I have built with configure/make so far were set up so that making the main project ensures all dependencies get build and installed before the main project is being built. That is the whole point of using built tools after all no? I am guessing this is a misunderstanding...

... there should be a libcoinasl.0.0.0.dylib in ThirdParty/ASL/.libs

Almost, the contents of Couenne/build/ThirdParty/ASL/.libs are:

I have the feeling that you must have modified something

Not that I can tell, I only did what I specified in my first post. Especially the get.ASL script was left as is...

Looks like ThirdParty/ASL was build using a more recent version of ThirdParty/ASL than the one supported by Counene

In my case the get.ASL script has a hard-coded address to download from: https://www.coin-or.org/BuildTools/ASL/${coinasl}.tgz with coinasl being set to solvers-20180528.

svn info from the ASL directory gives me:

svn info
Path: .
Working Copy Root Path: /Users/admin/Couenne/ThirdParty/ASL
URL: https://projects.coin-or.org/svn/BuildTools/ThirdParty/ASL/trunk
Relative URL: ^/ThirdParty/ASL/trunk
Repository Root: https://projects.coin-or.org/svn/BuildTools
Repository UUID: 51d96b7d-ff12-0410-af32-c88872dafd03
Revision: 4042
Node Kind: directory
Schedule: normal
Last Changed Author: stefan
Last Changed Rev: 4022
Last Changed Date: 2019-09-20 15:33:59 +0200 (Fri, 20 Sep 2019)

So indeed the svn repo seems to point to trunk instead of stable/1.4, however, the get.ASL file found at https://projects.coin-or.org/svn/BuildTools/ThirdParty/ASL/stable/1.4 has the same address for downloading the source... The only difference are the following lines:

it might be that a SoPlex version different than the one excepted by ThirdParty/SoPlex was used Indeed I had downloaded the source for version 2.0.1 instead of 2.0.0 for some reason.

I now manually changed the ASL url with svn switch https://projects.coin-or.org/svn/BuildTools/ThirdParty/ASL/stable/1.4 in Couenne\ThirdParty\ASL and ran the new get.ASL script again. I also got the correct code for SoPlex and reran configure and make, but now I am getting a bunch of undefined symbol errors from soplex... config.log make.log

Now I am running configure again with --without-soplex, I'll get back when this is done...

svigerske commented 4 years ago

The get.ASL script, which just downloads the ASL code, is indeed essentially the same between trunk and stable/1.4 of ThirdParty/ASL. But other files in this directory are much different. I don't really see how you got to the trunk of ThirdParty/ASL when using stable/0.5 of Couenne (Couenne/trunk points to trunk of ThirdParty/ASL, though - that could be fixed).

Looks like SoPlex lib couldn't be linked because some dependencies (GMP, zlib) are not linked in at the same time. Maybe that is caused by -single_module. I don't think there is much of an advantage to have SoPlex in Couenne, so skipping wouldn't hurt.

mlangiu commented 4 years ago

Yep after dropping SoPlex again it all works, make finishes without errors as well as make install and make test. So first off: Thanks for the help, much appreciated!

Remains to be answered why the urls to the dependencies are not set correctly, I tried issuing for d inls; do cd $d; svn info | grep ThirdParty; echo; cd ..; done In the ThirdParty directory and got:

Working Copy Root Path: /Users/admin/Couenne/ThirdParty/ASL
URL: https://projects.coin-or.org/svn/BuildTools/ThirdParty/ASL/stable/1.4
Relative URL: ^/ThirdParty/ASL/stable/1.4

Working Copy Root Path: /Users/admin/Couenne/ThirdParty/Blas
URL: https://projects.coin-or.org/svn/BuildTools/ThirdParty/Blas/releases/1.4.6
Relative URL: ^/ThirdParty/Blas/releases/1.4.6

Working Copy Root Path: /Users/admin/Couenne/ThirdParty/HSL
URL: https://projects.coin-or.org/svn/BuildTools/ThirdParty/HSL/releases/1.5.6
Relative URL: ^/ThirdParty/HSL/releases/1.5.6

Working Copy Root Path: /Users/admin/Couenne/ThirdParty/Lapack
URL: https://projects.coin-or.org/svn/BuildTools/ThirdParty/Lapack/releases/1.5.6
Relative URL: ^/ThirdParty/Lapack/releases/1.5.6

Working Copy Root Path: /Users/admin/Couenne/ThirdParty/Metis
URL: https://projects.coin-or.org/svn/BuildTools/ThirdParty/Metis/releases/1.3.5
Relative URL: ^/ThirdParty/Metis/releases/1.3.5

Working Copy Root Path: /Users/admin/Couenne/ThirdParty/Mumps
URL: https://projects.coin-or.org/svn/BuildTools/ThirdParty/Mumps/releases/1.5.5
Relative URL: ^/ThirdParty/Mumps/releases/1.5.5

Working Copy Root Path: /Users/admin/Couenne/ThirdParty/SCIP
URL: https://projects.coin-or.org/svn/BuildTools/ThirdParty/SCIP/releases/1.3.0
Relative URL: ^/ThirdParty/SCIP/releases/1.3.0

Working Copy Root Path: /Users/admin/Couenne/ThirdParty/SoPlex
URL: https://projects.coin-or.org/svn/BuildTools/ThirdParty/SoPlex/releases/1.4.2
Relative URL: ^/ThirdParty/SoPlex/releases/1.4.2

While the Dependencies state:

ThirdParty/Blas   https://projects.coin-or.org/svn/BuildTools/ThirdParty/Blas/stable/1.4
ThirdParty/Lapack https://projects.coin-or.org/svn/BuildTools/ThirdParty/Lapack/stable/1.6
ThirdParty/HSL    https://projects.coin-or.org/svn/BuildTools/ThirdParty/HSL/stable/1.5
ThirdParty/ASL    https://projects.coin-or.org/svn/BuildTools/ThirdParty/ASL/stable/1.4
ThirdParty/Mumps  https://projects.coin-or.org/svn/BuildTools/ThirdParty/Mumps/stable/1.6
ThirdParty/Metis  https://projects.coin-or.org/svn/BuildTools/ThirdParty/Metis/stable/1.3
ThirdParty/SoPlex https://projects.coin-or.org/svn/BuildTools/ThirdParty/SoPlex/stable/1.7
ThirdParty/SCIP   https://projects.coin-or.org/svn/BuildTools/ThirdParty/SCIP/stable/1.8

The same appears to be the case for the coin-or dependencies... Seems strange to me but I don't know enough about svn to see how this happened.

I guess one could simply use the Dependencies file and enter all subdirectories and update them manually, sth like

while IFS="" read -r p || [ -n "$p" ];
  do
  d=$(echo $p | cut -f1 -d' ');
  url=$(echo $p | cut -f2 -d' ');
  echo entering $d...;
  cd $d;
  svn switch $url;
  cd ..;
  echo;
done < Dependencies

But the question is why svn sets up the wrong urls in the first place...

svigerske commented 4 years ago

svn looks at the svn:externals property, which is out of sync with the Dependencies file. It would be normal if the svn:externals were pointing to releases from the stable branch that is mentioned in the Dependencies file, but the svn:externals actually point to releases from older stable branches. I don't know anymore whether that is intentional or not.

tkralphs commented 4 years ago

That is really strange about the dependencies. The svn:externals does match Dependencies in stable/0.5 (see https://projects.coin-or.org/Couenne/browser/stable/0.5). For releases, the svn:externals would be changed to all releases of other projects, but the Dependencies file would be left as stables to allow automatic updating to latest released versions of dependencies with a script.

The release versions you are showing for third party projects doesn't match the set of versions for any Couenne release, so this is almost surely not a set of versions you could get from just doing a straignt SVN checkout. I really wish I could unravel this mystery, as I'm curious myself. This set of versions does precisely match the versions of dependencies that go along with the last Ipopt release made in SVN (https://projects.coin-or.org/Ipopt/browser/releases/3.12.9/). About the only thing I can figure is that you somehow checked out Ipopt 3.12.19 first and then used those version when building Couenne or something. Anyway, glad it's fixed!