search

Project-Platypus / Platypus

A Free and Open Source Python Library for Multiobjective Optimization
GNU General Public License v3.0
553 stars 152 forks source link

More parallel processes than population size #217

Closed fabmid closed 8 months ago

fabmid commented 8 months ago

Hi, I noticed with the experimenter in parallel that the ProcessPoolEvaluator runs maxium as much parallel processes as that the population size. So in case population_size=20 and parallel processes=40, it still runs only 20 processes in parallel. Is this behaviour by design of the ProcessPoolEvaluator? Anythign to adapt that it can run 2 generations (2x20=40) in parallel? Thanks for your suport and this wonderful framework! Best Fabian

dhadka commented 8 months ago

No, multiple generations can't be evaluated in parallel.

This is just the design of evolutionary algorithms. The current generation is used to produce the offspring in the next generation, so we must fully evaluate the current generation before moving on to the next.

If you want to use all available parallel processes, you essentially need to increase the population size.

(There are other parallelization strategies for evolutionary algorithms, like the "island model", you could also consider. The island model lets you run multiple algorithms side-by-side. However, this library does not provide this capability out of the box.)

fabmid commented 8 months ago

Thank you for your fast reply and the clarification.