Native CBC interface?

[dginev]

Hi @jcavat , and cross-posting to @oddg and @xiaowei-routific .

I recently encountered significant performance overhead when using the CBC solver via "rust-lp-modeler" and realized a chunk of that is likely due to using system calls, rather than the native ABI interface.

I next spotted a repository, coin-or-cbc-sys which has used rust-bindgen to generate Rust bindings for CBC. There is an obvious synergy here, in my opinion, where the lp-modeler crate could at the least have a feature attribute that allows using CBC via the native interface, avoiding the syscall overhead (and potentially finding other avenues for speedup).

I have very little experience with CBC itself, and only recently found a need for an LP component in a larger application. As it happens, I have hundreds to thousands of small (~10 to 20 constraints) problems, so any overhead in the call to CBC itself becomes very noticeable.

Also, while I don't have much CBC experience, I have developed a couple of Rust wrappers over C toolkits before, notably one for libxml, so could share some experience from that if relevant. E.g. I do find the cbc-sys repo a little to customized for my taste, since I am a linux user I would have been satisfied to simply require the development headers (e.g. coinor-libcbc-dev on Ubuntu/travis CI) and linked the bindgen code against them.

Edit: to give some concrete numbers, a basic test runs with ~5 simple/easy constraints per LP problem, and sees a second of extra runtime spent on every 110 problems, or 9ms per problem. That doesn't sound like much, but given these problems are near-trivial it feels like it can be a tenth of that runtime if I was to write a (super-naive and crude) solver equivalent in Rust myself. I feel it's quite an unusual performance complaint to make, so maybe I should just write that and see if my claim is actually accurate...

[dginev]

And maybe I should also cc @taktoa who has created rust-clp, as well as @likr who seems to have done the same at clp-rs for COIN-OR's simpler purely linear programming solver Clp. Clp seems to be closest to what I am in need for in my current work, and calling it natively using the lp_modeler data structures and API would be quite elegant. I'll invest some time to experiment in that direction, and I am already quite curious to hear everyone's thoughts on whether there could be a clean organization of crates.io modules that makes this quite easy and direct for users and developers alike...

[jcavat]

Hi @dginev and thanks for your remarks. I've a small experience with CBC. Do you think we can reduce syscalls with parameters or this is due to the write lp file format/read file format step ? If so, maybe we can bypass writing a file on the disk. Maybe, this is not specific to CBC. If this is specific to CBC we can change the way we create the CBC solver. First, we can create a new CBC implementation that we can compare with the current one.

[jbx1]

I would also like to have the possibility to call the libraries natively instead of generating a temporary files, because my requirement is also to generate hundreds (potentially thousands) of LPs in different scenarios. Having a native binding directly might also bring another benefit, using the same basis but running it again "pre-warmed" for a different objective function (maybe you want to find the pareto frontier of the variables you are interested in), or maybe you want to incrementally add more constraints or variables without rebuilding the whole thing.

[jcavat]

We can imagine a NativeSolverTrait extending SolverTrait or maybe just a NativeCbcSolvert implementing SolverTrait with more features.

[carrascomj]

Hi, I would also be very interested in a native solution. Do you have any news about it @jcavat @dginev ?

[EdorianDark]

Since there is now a native bakend for Coinor CBC in master, I think that this can be closed.

[dginev]

Very cool! I'll give the new native interface a stress test in my own application and report back, but I suspect this is already a very workable solution.

[carrascomj]

@dginev this is already a very workable solution.

Sadly, I have tested on some stuff of my own (cargo test on run-native branch here) and it doesn't report the correct solution.

I will make another PR to address one of the errors: subtraction operations are not properly handled. It solves it for some problems but the solution of my current application is still wrong.

[jcavat]

close by #49