Solve multiple problems (different models) in parallel

[uurriola]

Hi,

I work in a parallel environment in python and I want to solve separate/different models in each of these threads/processes.

The issue is that I get different results when solving sequentially and in parallel. Looks like call to "solve" is not thread-safe (not even process-safe!). Do you have any insight about that?

I found this thread from 2013 saying solving multiple models is possible but depends on the way CBC is called: Thread safety

On another thread, someone (John Forrest) pushed an update to allow parallel use with ORTools : thread safe usage of CBC.

Alos, @h-g-s , you mentioned a year ago that CBC itself was not 100% thread-safe (in this issue), is that still true?

I can try to create a small example if necessary. Thx

[h-g-s]

Hi,

Solving different problems in parallel using python multiprocessing should be fine, each process will have a sequential CBC running.

BUT, normally parallel code is non-deterministic. Unless you are completely sure that no other process (not even the OS GUI/CLI) is running and your OS scheduler is very well behaved you'll have a code that is non-deterministic. Some solvers, like CPLEX, use very complicated techniques (that significantly degrade the performance) to ensure deterministic results while running in parallel. In CPLEX, the fastest parallel mode is called "opportunistic" and is not deterministic at all.

CBC can be used to solve one MIP in parallel, but this is not your case, I think.

[uurriola]

Hi,

First of all, thank you for your answer. I don't really get what part of the algorithm is not deterministic here... Shouldn't it work the exact same way as serial program ? I create multiple processes (only one model per process) and then run CBC through MIP, setting number of threads to 1 for each one of my models.

If you have any link or source on this subject, I'd really appreciate.

Thank you again

[tkralphs]

I was also a little bit confused by @h-g-s's comments. The statements above apply to the opportunistic version of parallel branch and bound that Cbc uses if you run a parallel version of Cbc, but I agree that the use of Cbc as you are describing it should (in principle) be deterministic. In coin-or/Cbc#223 that you point out, the person who opened the issue is also asking about two separate serial processes, not parallel Cbc, just as you are here. There, too, @h-g-s responded with an answer that looks like it applies only in the case of using parallel Cbc. @h-g-s can you clarify?

Anyway, I guess the question still stands. If you don't object, this is really a question about Cbc and I could transfer the issue over to there. In the meantime, perhaps @jjhforrest has some thoughts?

[h-g-s]

Hi @uurriola and @tkralphs . I am assuming that each execution is sequential but after finishing the execution they update some global information and other processes start. If the results of the optimization do not matter during the execution and they are only used after everything is finished then it really should be deterministic...

@uurriola , could you provide some not very big example where this happens ?

[h-g-s]

@uurriola , maybe if you could provide a package with your mips saved in mps/lp, we could check if there is some memory corruption bug that happens when solving one of the mips that is perhaps responsible corrupting the memory and affecting the execution flow

[uurriola]

I tried to reproduce without success today. Let's close this

[shaurakar]

Hello Everyone,

I am also facing exactly the same issue but in c++. I use CBC solver embedded in GOOGLE OR (c++ platform). I am using it for solving the mixed integer linear programming problem. I am following the sample code shown in the Google OR website https://developers.google.com/optimization/mip/integer_opt

However, I have a bunch of threads(pthreads)/threadpool that are trying to call solver (CBC) simultaneously. All of the threads have thread local data, and hence are calling the solver on their local data simultaneously. This also means that I am ensuring that the constraints, MPSolver, etc are all thread local (none of them are global). Objective is to solve multiple MILP problems in parallel. Each problem is solved by a thread by calling its own CBC solver (thread local).

Problem: If 'n' threads call the solver simultaneously, i see that the solver reports "No Solution Found" for all the thread local datasets. However, if the whole process is sequential i.e. different datasets are solved one after another by limiting the number of threads to 1 (means no multiple cbc solvers, only one), then I get perfect optimal solution for each and every datasets. Now, this happens only when the time limit has been set to a finite value. The time limit is set to 5s using the api solver.SetTimeLimit(). I can't avoid setting the time limit because there can be cases where it might take huge amount of time to get the optimal solution. Different threads can have different data sets which can be varying in size (number of constraints and number of variables). 'n' varies from 2 to 32

Note: Just to re-clarify I am trying to invoke simultaneously 'n' solver calls using 'n' threads (each thread calls it's own solver.solve()). I am not trying to distribute the work of a single solver among 'n' threads. All I am trying to do is to run multiple CBC solvers in parallel and each solver is being handled by a different thread.

I posted the same question in stack overflow nearly 2 months back. https://stackoverflow.com/questions/61867301/running-multiple-instances-of-google-or-solver-cbc-results-in-no-solution-foun Thanks

[dlaehnemann]

It seems like there is a workaround for this problem when you build from source, and a solution in the making: https://github.com/coin-or/Cbc/issues/332#issuecomment-786570533