MPI provides 'immediate' procedure calls that return a request instead of waiting for completion of the operation to complete (nonblocking send/recv, nonblocking collective operations, request-based RMA). The provided request is then used to test/wait for completion. Request management in application can be tedious, esp in multi-threaded environments. For example, some runtimes (e.g., PaRSEC) enqueue requests into a queue from which they are eventually taken and passed to MPI_Testsome to check for completion of the associated operation. Enqueuing and dequeuing may require additional synchronization and data movement.
Esp for small messages, there is a good chance that the operation has completed immediately locally by the time the initiating procedure call returns (e.g., if sufficient resources are available and the eager protocol is used). It would be possible to immediately test for the completion of this operation before enqueuing it into the list of outstanding operations that are eventually passed to MPI_Testsome. However, if the operation has not completed the call to MPI_Test will likely trigger the progress engine, which incurs additional overheads and synchronization.
Proposal
Introduce a new handle constant MPI_REQUEST_EMPTY that may be returned from a non-blocking operation. When passed to a test or wait function, the request is treated as complete and not cancelled. This can be safely done for nonblocking sends, nonblocking collectives, and RMA operations since their status is always empty. Thus, for these operations the introduction of MPI_REQUEST_EMPTY is backwards-compatible.
Nonblocking receive operations are a corner case since they are the only operations whose status contains operation-specific information. The use of MPI_REQUEST_EMPTY for receive operations would have to be enabled explicitly, e.g., via an info key "mpi_recv_req_may_be_empty", by applications that do not care about the status object and want to test for completion outside of MPI_Test. Querying the status of MPI_REQUEST_EMPTY will always return an empty status.
Some implementations already use an internal object that provides this facility (e.g., some parts of Open MPI may return a pointer to ompi_request_empty as the request handle). Implementations are free to never return MPI_REQUEST_EMPTY if they are unable to do so.
Impact on Users
Having a standardized way to check for completion of operations without invoking MPI_Test may reduce the required synchronization in multi-threaded applications. Applications that do not test for this handle are not impacted since the MPI_REQUEST_EMPTY behaves like any distinct request for the operations mentioned above.
Problem
MPI provides 'immediate' procedure calls that return a request instead of waiting for completion of the operation to complete (nonblocking send/recv, nonblocking collective operations, request-based RMA). The provided request is then used to test/wait for completion. Request management in application can be tedious, esp in multi-threaded environments. For example, some runtimes (e.g., PaRSEC) enqueue requests into a queue from which they are eventually taken and passed to
MPI_Testsometo check for completion of the associated operation. Enqueuing and dequeuing may require additional synchronization and data movement.Esp for small messages, there is a good chance that the operation has completed immediately locally by the time the initiating procedure call returns (e.g., if sufficient resources are available and the eager protocol is used). It would be possible to immediately test for the completion of this operation before enqueuing it into the list of outstanding operations that are eventually passed to
MPI_Testsome. However, if the operation has not completed the call toMPI_Testwill likely trigger the progress engine, which incurs additional overheads and synchronization.Proposal
Introduce a new handle constant
MPI_REQUEST_EMPTYthat may be returned from a non-blocking operation. When passed to a test or wait function, the request is treated as complete and not cancelled. This can be safely done for nonblocking sends, nonblocking collectives, and RMA operations since their status is always empty. Thus, for these operations the introduction ofMPI_REQUEST_EMPTYis backwards-compatible.Nonblocking receive operations are a corner case since they are the only operations whose status contains operation-specific information. The use of
MPI_REQUEST_EMPTYfor receive operations would have to be enabled explicitly, e.g., via an info key"mpi_recv_req_may_be_empty", by applications that do not care about the status object and want to test for completion outside ofMPI_Test. Querying the status ofMPI_REQUEST_EMPTYwill always return an empty status.Changes to the Text
See https://github.com/mpi-forum/mpi-standard/pull/959
Impact on Implementations
Some implementations already use an internal object that provides this facility (e.g., some parts of Open MPI may return a pointer to
ompi_request_emptyas the request handle). Implementations are free to never returnMPI_REQUEST_EMPTYif they are unable to do so.Impact on Users
Having a standardized way to check for completion of operations without invoking
MPI_Testmay reduce the required synchronization in multi-threaded applications. Applications that do not test for this handle are not impacted since theMPI_REQUEST_EMPTYbehaves like any distinct request for the operations mentioned above.References and Pull Requests
Pull Reqeust: https://github.com/mpi-forum/mpi-standard/pull/959