HiGHS solver integration

[samiit]

Hi,

Thanks for this package. I am still exploring it but things look familiar to my experience with JuMP (Julia). Is there any documentation on how to add additional solver support? I would like to add the HiGHS solver, which I did for PuLP a while ago. Of course, here there is a possibility for direct integration with its C++ API. Can someone suggest how this could be done?

Thanks, Sam

[rschwarz]

This is also of interest to me, so I'm happy to help.

I understand that python-mip uses the cffi library to make calls to C libraries. With that library, you can specify the functions you want to call by providing the C header as a string (see here for how that's done for Cbc) and loading the shared library, which is then used as if it where a Python module.

So it would be easiest to use the C interface to HiGHS, which is also how the Julia wrapper is implemented (so it should be well tested).

Not all features of python-mip are currently available with HiGHS (eg lazy constraint callbacks), but a wrapper would still be very valuable now!

[samiit]

Thanks for the pointer Robert @rschwarz . I will check and see what I can do.

[sebheger]

The way @rschwarz is describing is exactly the way how it should be done. I am also more than happy to support you @samiit.

[jajhall]

Not all features of python-mip are currently available with HiGHS (eg lazy constraint callbacks), but a wrapper would still be very valuable now!

Thanks for your interest in developing this interface to HiGHS. MIP callbacks and other features are on the stack of enhancements to be implemented in the near future.

[ryanjoneil]

👋 Checking in to see if this is in development yet and if you need contributions.

[jajhall]

Much of what you want is very much in development. Specifically, MIP callbacks and a thin Python interface to the C++ API, coupled with being able to pip install HiGHS. This will eliminate the need to go through the C interface. I don't believe we can automatically generate a Python interface to the C API since some HiGHS parameter lists involve passing by reference, something that I understand is not possible with Python. Happy to be corrected

[sebheger]

@ryanjoneil From a python-mip side it is not under development. Very happy for any contributions

[sebheger]

Much of what you want is very much in development. Specifically, MIP callbacks and a thin Python interface to the C++ API, coupled with being able to pip install HiGHS. This will eliminate the need to go through the C interface. I don't believe we can automatically generate a Python interface to the C API since some HiGHS parameter lists involve passing by reference, something that I understand is not possible with Python. Happy to be corrected

@jajhall python-mip is using cffi to directly interact with the C API (and only with the C API) of a local existing binary. It doesn't provide any benefit for us if you offer a thin python interface around the C++ API as it doesn't match the core design. You need "C-binaries" for CBC (self-build or the ones distributed with python-mip) or gurobi (either downloaded or via pip-installed) and indirectly the header files (are at the moment part of the python-mip repo). If the pip-installed HiGHs would come with binaries that can be accessed via C would help most. The main issue at the moment is, that the C interface of CBC should be considered to be unmaintained and the build process of CBC could be considered as "toxic" when not using coin-brew. Especially the C-Interface is not part of the official version build.

cffi is by the way able to handle input/output of C-functions during runtime. You could store even complex things like the created model from C in a python variable. The "only" requirement is that everything could be executed/access through a C-function (by the respective wrapped function).

[jajhall]

We now have HiGHS as pip install on Linux

[rschwarz]

FYI: I'm planning on working on this during January.

[sebheger]

@rschwarz Would be nice if we could have an exchange on this