search

DEAP / deap

Distributed Evolutionary Algorithms in Python
http://deap.readthedocs.org/
GNU Lesser General Public License v3.0
5.86k stars 1.13k forks source link

request for enhancement of operational readiness #681

Open cherusk opened 1 year ago

cherusk commented 1 year ago

I'm intending to include deap as a backend for the godon project.

I've only found references to scoop for operationally parallelizing the framework.

Hence the question, are there near or mid term plans on your end to extend the set of supported orchestration engines? Examples could be docker swarm, kubernetes or dask.

If not, could you give me pointers of what are be stumbling blocks to do so?

cherusk commented 1 year ago

Citing https://github.com/markcoletti:

"I see that you are interested in a distributed EA framework, which is a functionality that DEAP currently does not have. (And which is a frequently requested feature for DEAP.) You might be interested in Library for Evolutionary Algorithms in Python (LEAP) as it natively supports Dask for distributed evaluations. It has been successfully deployed at scale on the Summit supercomputer. Full disclosure: I am one of the LEAP developers. You can find LEAP here: https://github.com/AureumChaos/LEAP/tree/develop"

cherusk commented 1 year ago

LEAP looks promising.

As a terrific reference, the optuna framework dev team does offer a great option for dask parallelization integration, even part of their latest pre-release by now:

https://github.com/optuna/optuna-examples/blob/main/dask/dask_simple.py