RobotLocomotion / drake

Model-based design and verification for robotics.
https://drake.mit.edu
Other
3.35k stars 1.27k forks source link

Provide ability to call SolveInParallel with program generators. #22226

Open AlexandreAmice opened 14 hours ago

AlexandreAmice commented 14 hours ago

Since the introduction of SolveInParallel in #21957 , there have been several PRs which try to match this pattern (e.g. #21705, #21351, #22222). In these cases, it is memory intensive to generate all the programs at once in memory, however it is easy to say decide what the ith program is. When solving MathematicalProgram in parallel there are a couple of subtleties that are easy to forget/miss when implementing. 1.) Setting CommonSolverOptions.kMaxThread = 1 2.) Skipping over programs which are not thread safe and solving these in a second pass in serial. 3.) Managing the number of solvers which are created by the threads.

Managing these complexities was a major motivator for implementing SolveInParallel and why it is generally a bad idea to implement the parallel for loop manually each time.

To avoid this potential for implementation errors, I propose adding a new overload

std::vector<MathematicalProgramResult> SolveInParallel(
    const std::function<std::unique_ptr<MathematicalProgram>(int64_t, int64_t)>&
        prog_generator,
    const std::function<std::optional<Eigen::VectorXd>(int64_t, int64_t)>&
        initial_guesses_generator,
    const std::function<std::optional<SolverOptions>(int64_t, int64_t)>&
        solver_options_generator,
    const std::function<std::optional<SolverId>(int64_t, int64_t)>&
        solver_ids_generator,
    const int64_t range_start, const int64_t range_end, Parallelism parallelism,
    bool dynamic_schedule)

which all parallel for loops of MathematicalProgram should route through.

I am willing to make this function internal, but I think it could be beneficial to have this in the public API.

See draft PR #22225

cohnt commented 6 hours ago

+1 to implementing this feature -- thanks for starting to work on it!