Add functional argument to CBC_CMD/COIN_CMD to allow setting random seed

bsaunders23 commented 1 year ago

Describe the new feature

Expose setting of RandomS(eed) option for CBC/COIN-OR solvers as direct argument to solver instantiation API (pulp.apis.coin_api.PULP_CBC_CMD, pulp.apis.coin_api.COIN_CMD). Happy to work on the PR here (it's a fairly trivial change), just didn't want to launch into it before getting the :thumbsup:.

It's important to be able to control reproducability of results in testing consistency of solutions or isolating the relative impact of changing certain parameters in the MIP problem, however identifying the RandomS(eed) CL argument to pass to the solver can be a bit of a round-about exercise for users who aren't as familiar with the actual solver implementation. Adding the ability to set RandomSeed directly from the API creation will directly expose this ability to end users, which represents a general improvement in the solution.

Additional Notes:

Think we can forego settings for RandomC since this defaults to the static value of -1 in the current CBC solvers in pulp.solverdir.cbc..
Came across an example where a user was facing challenges in reproducability and had to set a different arg for nthreads (https://github.com/coin-or/Cbc/issues/315). I've tested reproducability with my own problem and found consistent results with setting just RandomSeed, but might be something to investigate further if there is some potential risk to reproducability in certain scenarios with just setting RandomSeed.

Currently achievable by passing to solver via options:

import pulp
cbc_solver = pulp.PULP_CBC_CMD(keepFiles=False,
             threads=8,
             #Set random seed to ensure reproducability, passes as command-line arg to solver
             options= ["RandomS 20"]
             )

Additional info

Please answer these questions before submitting your feature request.

Is your feature request related to an issue? Please include the issue number.

Couple examples I could find with a quick search on this git (not to say there aren't other ones out there)

Does this feature exist in another product or project? Please provide a link.

Not to my knowledge

bsaunders23 commented 1 year ago

@Franken151 interesting results! I'm not quite sure I know myself what's happening, but based on Total time vs Wall time it seems like most (or all) of the solution is being solved in the cutting planes vs actual branch and bounding (note enumerated nodes = 0). If you want to share the full output I'd be happy to have a look over it, but I'm betting that this probably has something to do with being solved by the initial heuristics vs actual node exploration.

You could maybe look at explicitly disabling randomizedRounding, details here. I think if we look at the full output and see exactly what element of the cuts/heuristics is changing in iteration count.

mip:randomizedrounding (randomizedRounding)
      Whether to try randomized rounding heuristic

      off    - disabled
      on     - use every node in the tree
      both   - 
      before -

Franken151 commented 1 year ago

False Alarm. It's on me.... It's somewhere in the model creation process, which involves some data pre-processing which should be completely deterministic, but after diff'ing the created .lp files from pyomo from sequential executions with no changes made, there are slight deltas that I'll need to dig in on and find the source of the non-deterministic behavior. When I run repeatedly on a single .lp file all is as expected with the CBC output. Thanks for your prompt reply!

MLopez-Ibanez commented 1 year ago

This is not only important to control reproducibility, but changing the random seed may help solving a problem faster: https://doi.org/10.1287/opre.2013.1231

Most (all?) MIP solvers are stochastic (but sometimes they have a default random seed).

coin-or / pulp