eth-cscs / DLA-Future

DLA-Future
https://eth-cscs.github.io/DLA-Future/master/
BSD 3-Clause "New" or "Revised" License
64 stars 14 forks source link

Use pika's transform_mpi and polling support #1125

Open biddisco opened 5 months ago

biddisco commented 5 months ago

Fixes #536.

This is a squashed commit containing multiple changes

completion_modes: pika supports different completion modes that may be used as an alternative to the dlaf:: transformMPI mechanism that uses yield_while to wait on an MPI request.

The completion modes may be set via the environment variable PIKA_MPI_COMPLETION_MODE= which by default will select the one chosen by pika/dlaf developers known to give good results across a broad range of use cases.

polling: The pika polling loop may test for one or multiple request completions on each iteration through the scheduling loop the environment var PIKA_MPI_POLLING_SIZE= (default 8) may be used to vary the polling size (typically the default value can be used without any need to play with this value)

mpi pool: pika will create the mpi pool if the completion mode has the pool flag set, the user needs only to call the pool create function during the pika::init setup phase. Cleanup of the pool on shutdown will also be handled automatically

The user should use pika::mpi::pool_name instead of raw "mpi", mpi pool management has been deferred tom pika::mpi

Change: the transform mpi code does not need to return an MPI_SUCCESS value, the return value from mpi_transform has been removed to simplify code and an error is set using senders set_error if any mpi call fails. Should mpi_transform calls thnat return other value be required, this code can be reinstated.

biddisco commented 5 months ago

Note that this PR depends on the mpi_polling pr in pika so probably won't work with pika master

msimberg commented 5 months ago

I've just merged https://github.com/pika-org/pika/pull/1102 so the changes are on pika main now.