Solve Mathematical Program in Parallel

[AlexandreAmice]

Is your feature request related to a problem? Please describe. There are many uses cases for solving multiple mathematical programs in parallel. Examples include starting trajectory optimization from multiple initial guesses, certifying that no collision will occur in a region of C-Space, and parallelizing the solution to quasi-convex optimization programs.

Currently this is possible in C++, albeit it requires writing the parallelized code on your own. Having a method which handles this procedure would be a nice addition for C++ users. For Python users, this is currently not possible due to the GIL and the inability to pickle MathematicalProgram.

Describe the solution you'd like Provide a parallelized Solve function that is bound in Python would provide an elegant solution for both users. This would require resolving #10320.

Describe alternatives you've considered Enabling the pickling of MathematicalProgram so that Python users can use the multiprocessing or similar libraries would allow Python users the ability to get around the GIL. However, this could be quite slow especially for large mathematical programs.

[nepfaff]

I support the pickling of MathematicalProgram idea for Python parallelism. In particular, I'm interested in evaluating AugmentedLagrangianNonsmooth in parallel in Python (use with Nevergrad black-box optimization). Pickling AugmentedLagrangianNonsmooth requires pickling MathematicalProgram.

Edit after discussion with Alex: Not reasonable for plant-dependent MathematicalPrograms...

[AlexandreAmice]

To those who come across this issue in the future, note that the current status is that many of the constraints in MathematicalProgram are not thread safe. Examples of these are all the constraints which interact with multibody plant and context.

This makes pickling a MathematicalProgram essentially impossible as it would require serializing the entire systems framework.

It may be possible to safely parallelize a convex MathematicalProgram

[RussTedrake]

@jwnimmer-tri -- could I ask you to weigh in here about whether it is currently safe to parallelize convex optimization solves? In #21770, it seems like you were ok with it (using the one-prog-per-core model)?

[jwnimmer-tri]

Let me check that we're all talking about the same thing.

The proposal here is for a new C++ function (also bound in pydrake) something like this?

/** Solves prog[i] into result[i], optionally using initial_guess[i] and solver_options[i] if given.
Uses at most parallelism cores, with static scheduling by default. */
std::vector<MathematicalProgramResult> SolveParallel(
    const std::vector<const MathematicalProgram*>& prog,
    const std::vector<const Eigen::VectorXd*>* initial_guess = nullptr,
    const std::vector<const SolverOptions*>* solver_options = nullptr,
    const Parallelism parallelism = Parallelism::Max(),
    bool dynamic_schedule = false);

Architecturally, I think this fine and good.

As to the thread-safety and correctness, that's a question to be judged of the implementation not the interface.

[AlexandreAmice]

Steps to implement this are 1) Land #21857 2) Add the member thead_safe to Constraint 3) Add the member thread_safe to MathematicalProgram 4) Implement SolveParallel