Gurobi dependency

[otaolafranc]

is it planned to get rid of the gurobi dependency for other open-source solutions? I am looking to use AlgoHex which has QGP3D as dependency, which at the same time has gurobi with can complexify the use of this great piece of soft. An issue regarding this has been also post it in algoHex for the same concernings () thanks in advance,

[mheistermann]

@HendrikBrueckler added support for coinor-bonmin in https://github.com/HendrikBrueckler/QGP3D/commit/e218c98463a596cfccec4f2689ef0d781a1f6da7

[otaolafranc]

Nice! I have saw some activity in this regard last few days. are the compilation instructions going to be updated? :D (anxious ahahah)

[HendrikBrueckler]

Sorry for taking so long to get around to this. There's indeed a tentative replacement for Gurobi, which comes with a small performance overhead in comparison, but that should not matter too much. Said replacement is used as a backup in case Gurobi could not be found on your system.

The additional dependencies are listed in the README. The coinor-based libraries should be not much of the problem, depending on your system you can probably get them via the systems package manager or install from source, both of which worked quite easily for me on Linux. A bit of a challenge might be Ipopt, because you need some third party libraries for that (e.g. HSL or MUMPS), but they do a good job at explaining how to install everything :)

[otaolafranc]

Sorry for taking so long to get around to this. There's indeed a tentative replacement for Gurobi, which comes with a small performance overhead in comparison, but that should not matter too much. Said replacement is used as a backup in case Gurobi could not be found on your system.

The additional dependencies are listed in the README. The coinor-based libraries should be not much of the problem, depending on your system you can probably get them via the systems package manager or install from source, both of which worked quite easily for me on Linux. A bit of a challenge might be Ipopt, because you need some third party libraries for that (e.g. HSL or MUMPS), but they do a good job at explaining how to install everything :)

Hello @HendrikBrueckler , first of all no need to be sorry, you are trying to help others use your work so thank you :) once that said, I am struggling a little bit with the dependencies. could you tell me if I am going the correct route?

right now I have done this:

#installDependencies
#######
#ipopt
#######
    sudo apt-get install gcc g++ gfortran git patch wget pkg-config liblapack-dev libmetis-dev
    sudo apt install coinor-libcgl-dev coinor-libcgl-doc coinor-libcgl1 coinor-cbc coinor-libcbc-dev coinor-libcbc-doc coinor-libcbc3 coinor-libclp-dev  coinor-libclp-doc coinor-libclp1 coinor-libosi-dev coinor-libosi-doc  coinor-libosi1v5 -y

    git clone https://github.com/coin-or/Ipopt
    cd Ipopt

    #cbc
    sudo apt-get install  coinor-cbc coinor-libcbc-dev

    #asl
        git clone https://github.com/coin-or-tools/ThirdParty-ASL.git
        cd ThirdParty-ASL
        ./get.ASL
        ./configure
        make
        sudo make install
    ######end asl

    #HSL
    git clone  https://github.com/coin-or-tools/ThirdParty-HSL
    cd ThirdParty-HSL
    cp $path/coinhsl-archive-2023.11.17.tar.gz .
    gunzip coinhsl-archive-2023.11.17.tar.gz
    tar xf coinhsl-archive-2023.11.17.tar 
    ln -s coinhsl-archive-2023.11.17 coinhsl
    ######end HSL

    #MUMPS Linear Solver
    git clone https://github.com/coin-or-tools/ThirdParty-Mumps.git
    cd ThirdParty-Mumps
    ./get.Mumps
    ./configure
    make
    sudo make install
    ######end MUMPS Linear Solver
    ./configure
    make
    sudo make install

#######
######end ipopt
#######

#######
#Bonmin
#######
git clone https://github.com/coin-or/Bonmin
cd Bonmin
./configure -C  
make  
make install

but during the make of Bonmin, it stops with the error:

../../../src/Interfaces/BonOsiTMINLPInterface.hpp:21:10: fatal error: OsiSolverInterface.hpp: No such file or directory 21 | #include "OsiSolverInterface.hpp" which makes me think that I should install Osi, Cgl, Cbc, Clp but from the page https://github.com/coin-or/Osi the command (I am using ubuntu based system): sudo apt-get install coinor-osi coinor-libosi-dev does not find the package. I tried with the second sudo apt install to install all the packages that made me think were relevant by name from here: https://packages.debian.org/search?keywords=coinor&searchon=names&suite=stable&section=all without much luck. any insight? nor cgl using sudo apt-get install coinor-cgl coinor-libcgl-dev (indicated here https://github.com/coin-or/Cgl)

[mheistermann]

Some coin-or packages in Debian are very outdated (e.g. ipopt, I'm guessing others), so mixing and matching current upstream packages of coin-or with them might not work well.

I've had some success with coinbrew in a container based build, maybe that'll work for you?

Btw, you'll only need either MUMPS or HSL (MUMPS is the freely licensed option), not both. EDIT: Just checked, coinbrew will also install mumps for you.

[otaolafranc]

hello, thanks for the answer, so just to be clear simply run:

wget https://raw.githubusercontent.com/coin-or/coinbrew/master/coinbrew
chmod u+x coinbrew
./coinbrew fetch Osi@master
./coinbrew build Osi
./coinbrew fetch Cbc@master
./coinbrew build Cbc
./coinbrew fetch Cgl@master
./coinbrew build Cgl
./coinbrew fetch Clp@master
./coinbrew build Clp
./coinbrew fetch Ipopt@master
./coinbrew build Ipopt
./coinbrew fetch Bonmin@master
./coinbrew build Bonmin

best regards

[mheistermann]

This is currently compiling for me (still waiting for it to finish, didn't test yet): https://github.com/cgg-bern/AlgoHex/blob/dev/dockerfile-without-gurobi/Dockerfile

(You could use it to build a container, or as instructions to build on your main system)

(Note that branch uses a shallow fork of QGP3D to address a cmake problem with eigen)

[mheistermann]

I have some build error at the end, seemingly caused by an out-of-date ipopt version pulled in by bonmin: https://github.com/coin-or/Bonmin/blob/master/.coin-or/Dependencies There's probably some way to override this, but i didn't check.

[otaolafranc]

hello, thanks for the answer, so just to be clear simply run:

wget https://raw.githubusercontent.com/coin-or/coinbrew/master/coinbrew
chmod u+x coinbrew
./coinbrew fetch Osi@master
./coinbrew build Osi
./coinbrew fetch Cbc@master
./coinbrew build Cbc
./coinbrew fetch Cgl@master
./coinbrew build Cgl
./coinbrew fetch Clp@master
./coinbrew build Clp
./coinbrew fetch Ipopt@master
./coinbrew build Ipopt
./coinbrew fetch Bonmin@master
./coinbrew build Bonmin

best regards

this normally worked for all of them. nevertheless, when compiling AlgoHex which compiles QGP3D does not detect Bonmin.... :/

[mheistermann]

Try setting environment variables / cmake variables to point to the right place. CMake finders are just heuristics that probably don't include the paths where this was installed. In my dockerfile, I set -D BONMIN_ROOT_DIR=/opt/coin-or (that's the location I manually passed to coinbrew; not sure what the default is).

In any way, you will probably run into the same problem with the downgraded ipopt (and if not, teach me your ways ;-) ), symptom is:

/app/external/QGP3D/src/MCQuantizer.cpp:866:27: error: no member named 'SetBoolValue' in 'Ipopt::OptionsList'
        bonmin.options()->SetBoolValue("hessian_constant", true);
        ~~~~~~~~~~~~~~~~~~^

I'm currently trying this to avoid the downgrade:

coinbrew fetch Bonmin@master --skip-update
coinbrew build Bonmin@master --skip-update --prefix=/opt/coin-or --parallel-jobs 8 ADD_FFLAGS=-fallow-argument-mismatch

[mheistermann]

Update: I successfully built Gurobi-free AlgoHex with https://github.com/cgg-bern/AlgoHex/blob/dev/dockerfile-without-gurobi/Dockerfile (Command was podman build --memory 128G -t algohex .; the default memory limit is not enough, but 128G is overkill, I think 16 or 32 should be fine).

Okay, now need to slightly change AlgoHex to actually use QGP3D even when Gurobi is not available (current version will skip quantisation). But that's out of scope for the issue here :)

For posteriority, these were the commands that gave me the right coin-or library versions:

coinbrew fetch https://github.com/coin-or-tools/ThirdParty-Mumps@3.0.5
coinbrew fetch Ipopt@3.14.13 --skip-update
coinbrew fetch Bonmin@master --skip-update
coinbrew build Ipopt@3.14.13 --verbosity 2 --skip-update --prefix=/opt/coin-or --parallel-jobs 8 --tests none 
coinbrew build Bonmin@master --verbosity 2 --skip-update --prefix=/opt/coin-or --parallel-jobs 8 --tests none 

I also did ln -s /opt/coin-or/include/coin-or /opt/coin-or/include/coin as a quick hack to satisfy the CBC finder that disagrees with the installed version on the subdirectory name.

[mheistermann]

Confirmed to work in the AlgoHex dev/dockerfile-without-gurobi branch :) Please try that one and continue discussing at https://github.com/cgg-bern/AlgoHex/issues/15 :)

[HendrikBrueckler]

With our newest update, our implementation now supports getting rid of IQP solvers altogether. Only Clp is still needed as an LP solver, the other dependencies are now optional.