Host communications interface can be used for inter-process and/or inter-node communications. It mimics MPI's interface without being tied to it and adapting it to a more C++-like style.
This interface is only suited for host-host communications and not for device-device. In the future, a device_communicator interface will be added to achieve the later task. The reason behind this separation is that device communication APIs (e.g nccl or rccl) rely on a host communicator (e.g. MPI) for initialization. In addition, there are cases in which plain variables need to be shared across peers without any fancy in-device buffer abstraction.
Thus, the purpose of the host communicator interface is to coordinate nodes and help device communicators
host_communicator_backend is an abstract class that serves as the entry point for a particular implementation. E.g MPI.
host_communicator_manager centralizes all known implementations of host_communicator_backend
host_communicator_backend provides a world communicator, an instance of host_communicator that communicates all peers in a cluster.
host_communicator can perform typical communication operations (send/receive/broadcast/scatter/gather...)
-- host_communicator can be sub-divided into smaller communication domains.
Also added span class, a pre C++20 implementation of std::span, which is used for passing buffers to host_communicator.
Once this PR is merged, we can start creating the MPI plugin
Host communications interface can be used for inter-process and/or inter-node communications. It mimics MPI's interface without being tied to it and adapting it to a more C++-like style.
This interface is only suited for host-host communications and not for device-device. In the future, a
device_communicator
interface will be added to achieve the later task. The reason behind this separation is that device communication APIs (e.g nccl or rccl) rely on a host communicator (e.g. MPI) for initialization. In addition, there are cases in which plain variables need to be shared across peers without any fancy in-device buffer abstraction.Thus, the purpose of the host communicator interface is to coordinate nodes and help device communicators
host_communicator_backend
is an abstract class that serves as the entry point for a particular implementation. E.g MPI.host_communicator_manager
centralizes all known implementations ofhost_communicator_backend
host_communicator_backend
provides a world communicator, an instance ofhost_communicator
that communicates all peers in a cluster.host_communicator
can perform typical communication operations (send/receive/broadcast/scatter/gather...) --host_communicator
can be sub-divided into smaller communication domains.span
class, a pre C++20 implementation ofstd::span
, which is used for passing buffers tohost_communicator
.Once this PR is merged, we can start creating the MPI plugin
The TODO in this PR needs to be modified once it has been merged with https://github.com/gigabit-clowns/xmipp4-core/pull/56