esa / pagmo2

A C++ platform to perform parallel computations of optimisation tasks (global and local) via the asynchronous generalized island model.
https://esa.github.io/pagmo2/
GNU General Public License v3.0
804 stars 159 forks source link

MPI Island class for cluster computing #563

Open montanaviking opened 6 months ago

montanaviking commented 6 months ago

I'd like to see an MPI Island class for cluster computing whereby islands can run and migrate on several machines to perform one optimization problem. This capability existed in Pagmo1x but does not yet exist in Pagmo2x. It doesn't have to use MPI specifically, but must allow optimization and decision vector migration between machines. I'm thinking of creating a Pagmo::Islandcluster by copying and modifying Pagmo::Island to include the use of sockets and serialization. At some point, I'd like to contribute a solution, but wondering if there's any documentation for Pagmo::Island internals than the source code? If someone has this documentation, could you kindly let me know? Thanks, Phil

montanaviking commented 6 months ago

Actually, I'd rather not use MPI or OpenMPI but rather retain use of Intel TBB and use socket code + serialization to create the multi-node island class because I think that TBB is faster than other solutions for multiprocessing and has other advantages as well. I'm looking at the Pagmo2 Island class source code but would like input and documentation regarding its internal interface to the parent archipelago for migration, if someone can assist me? I would use the present pagmo::island class as a starting point and template to create a new pagmo::island_multinode class that would enable migrations across machines via the machines' LAN network interface cards (NICs). This would certainly make Pagmo2 more exciting. Thanks, Phil