search

rust-or / good_lp

Linear Programming for Rust, with a user-friendly API. This crate allows modeling LP problems, and lets you solve them with various solvers.
https://crates.io/crates/good_lp
MIT License
216 stars 35 forks source link

Add support for CPLEX through the bindings to the C API #43

Closed mbiggio closed 3 months ago

mbiggio commented 3 months ago

I added support for the CPLEX solver through the bindings to the C API, which are provided by the cplex-rs crate.

The raw bindings are built at compile time by the cplex-rs-sys crate, checking the CPLEX_PATH env variable to determine the CPLEX installation path, or defaulting to a standard installation directory. cplex-rs provides safe bindings on top of the raw ones.

Calling CPLEX as a library allows to avoid the overhead of going through the IO and parsing for .lp and .sol files generation, and to customize the solver behavior by setting parameters in the Environment object.

I'm not sure however if it is possible to test this feature in the CI, due to the CPLEX proprietary licence. Would you be interested in considering this pull request? If yes, do you have suggestions on how to integrate it so that the CI is green?

Thanks!

lovasoa commented 3 months ago

would it be possible to test it on CI using the community edition ?

mbiggio commented 3 months ago

would it be possible to test it on CI using the community edition ?

Sure, I can give it a try

lovasoa commented 3 months ago

@mbiggio : it looks like you have a syntax error in your yaml

mbiggio commented 3 months ago

@mbiggio : it looks like you have a syntax error in your yaml

@lovasoa the yml should be fixed now. The approach I took is to run the cplex tests in a specific job, running on a docker image with CPLEX CE installed, hosted in a private docker repo.

I set up the variable docker_hub_private_repo to identify the docker repo, and the secrets necessary to log in and download the image. If no variable docker_hub_private_repo is present, the cplex tests are skipped.

I took the same approach in the cplex-rs repo, and also added a README.md there on how to setup the docker image with your own CPLEX CE installation.

I did that because in order to get the CPLEX CE installer, an account is needed on the IBM cplex website. Therefore I couldn't just download the installer from the CI.

Since I'm not distributing my own copy of the CPLEX CE, but just using in my CI pipeline, I think this approach should be fine in terms of licensing. However, I could not find the CPLEX CE license to confirm this.

Let me know what you think of this approach :)

lovasoa commented 3 months ago

I'm not a big fan on depending on a private docker repo. This means the CI could break without anything we could do.

Could we have a IBM_LOGIN and IBM_PASSWORD secrets in the repo, and use these directly, keeping everything open-source ?

mbiggio commented 3 months ago

I'm not a big fan on depending on a private docker repo. This means the CI could break without anything we could do.

Could we have a IBM_LOGIN and IBM_PASSWORD secrets in the repo, and use these directly, keeping everything open-source ?

You mean downloading directly the installer from IBM, by logging in with the secret credentials? I agree that would probably be the best solution, but I'm not sure IBM provides a way to programmatically download the installer, at least as far as I could see from their documentation/website

lovasoa commented 3 months ago

Actually, it looks like you don't even need to log in to download the files: https://ibm.ent.box.com/s/9gy0r3eue56ovwrelndqfvtjnxcu5amg

mbiggio commented 3 months ago

Actually, it looks like you don't even need to log in to download the files: https://ibm.ent.box.com/s/9gy0r3eue56ovwrelndqfvtjnxcu5amg

That would be awesome, but in the details section I see:

This shared link will be disabled on Jul 31, 2024.

lovasoa commented 3 months ago

Ok, I just uploaded it, I'll take the responsibility if anything happens: https://github.com/rust-or/good_lp/releases/download/cplex/cplex.bin

mbiggio commented 3 months ago

Ok, I just uploaded it, I'll take the responsibility if anything happens: https://github.com/rust-or/good_lp/releases/download/cplex/cplex.bin

Ok, thanks, I will update the PR tomorrow

mbiggio commented 3 months ago

@lovasoa : I added the installation of CPLEX from the link you provided in the last commit.

There is however one further fix I needed to do. The cplex-rs and lpsolve fetures are going to be incompatible, since both crates link statically to C-libraries defining some common symbols, and therefore generate some linker errors:

  = note: /usr/bin/ld: /opt/ibm/ILOG/CPLEX_Studio_Community2211/cplex/lib/x86-64_linux/static_pic/libcplex.a(_daxpy_lp64.o): in function `daxpy':
          _daxpy.c:(.text+0x0): multiple definition of `daxpy'; /home/runner/work/good_lp/good_lp/target/debug/deps/liblpsolve_sys-319d5ea6682c179f.rlib(myblas.o):/home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lpsolve-sys-5.5.0/lp_solve_5.5/shared/myblas.c:169: first defined here
          collect2: error: ld returned 1 exit status

In case the two features are simultaneously activated, I'am now generating a compile-time error message instead of a linker error, which is probably a little nicer, but this anyway means that compilation will --all-features will not work anymore.

There is no other solution I could foresee, lemme know whether this is acceptable or not

lovasoa commented 3 months ago

I think that's okay if it's well documented. An explicit note about --all-features in the readme, maybe...

lovasoa commented 3 months ago

We need to add information about cplex to the readme together with other solvers too.

lovasoa commented 3 months ago

Great ! Thank you @mbiggio !

mbiggio commented 3 months ago

Great ! Thank you @mbiggio !

You are welcome!

lovasoa commented 3 months ago

@mbiggio : looking at the code after the fact, I have a suggestion.

You may want to create a CplexSolver(CplexEnvironment) struct, in order to be able to use a custom environment and still use the nice default maximize(objective).using(CplexSolver(my_env)) syntax.

You can have a look at how it's done in the LpSolvers solver, which also takes a configuration option: https://github.com/rust-or/good_lp/blob/main/src/solvers/lp_solvers.rs#L21-L23

mbiggio commented 3 months ago

@mbiggio : looking at the code after the fact, I have a suggestion.

You may want to create a CplexSolver(CplexEnvironment) struct, in order to be able to use a custom environment and still use the nice default maximize(objective).using(CplexSolver(my_env)) syntax.

You can have a look at how it's done in the LpSolvers solver, which also takes a configuration option: https://github.com/rust-or/good_lp/blob/main/src/solvers/lp_solvers.rs#L21-L23

@lovasoa yes, makes sense. Will do the change, thanks