mpiwg-sessions / sessions-issues

3 stars 1 forks source link

Need to define a local variant of MPI_Session_finalize #23

Open hppritcha opened 2 years ago

hppritcha commented 2 years ago

As pointed out during preparation of the MPI 4 standard, the way MPI_Session_finalize is defined as potentially synchronizing:

MPI_SESSION_FINALIZE may be synchronizing on any or all of the groups associated
with communicators, windows, or  les derived from the session and not disconnected, freed,
or closed, respectively, before the call to the MPI_SESSION_FINALIZE procedure.

revealed some less than desirable complexities for applications using sessions in non-trivial ways.

There was some agreement that the asymmetry between the local nature of MPI_Session_init and potentially non-local behavior of MPI_Session_finalize was part of the problem. For MPI-5 it would be good to have an alternative, local method for cleaning up local MPI resources associated with a session.

abouteiller commented 1 year ago

I propose this would be named MPI_Session_free and have the same kind of meaning that MPI_COMM_FREE has for communicators (that is, it marks for deletion, deletion and message completion may be delayed to a later time).

RolfRabenseifner commented 1 month ago

I recommend that all, who want to work on this issue, should be completely familiar with the progress rule of MPI, the special rule for MPI_Finalize and MPI_Session_finalize, and the relation to the possibilities to correctly implement an MPI library. Especially if an MPI implementation does not use instant progress (though OS threads or an additional processor like the NIC), then the progress rule usually relies on that another MPI call will be issued by the MPI application, which is guaranteed through the rules about MPI_Finalize and MPI_Session_finalize. If you propose a "MPI_Session_free", then there may be still progress needed, but there may be no additional call to an MPI procedure. The discussion about MPI_Session_finalize has shown, that the knowledge about the progress rule was very rare in the MPI forum. I hope that the new MPI-4.1 Section 2.9 Progress helps to clarify the problems and consequences for implementations, if you propose such an "MPI_Session_free".

abouteiller commented 1 month ago

Thanks for the comment. The main use case for the non-synchronizing session closing (lets call it SESSION_FREE) is when used in conjunction with MPI_COMM_DISCONNECT to guarantee that the channels are flushed independently of the SESSION_FREE call (and thus no progress required during the session_free call).

In other cases (e.g. using MPI_COMM_FREE without care), calling SESSION_FREE would not progress to completion the required communication, as you highlighted. This is analogous to the MPI_COMM_FREE vs MPI_COMM_DISCONNECT/MPI_FINALIZE semantics difference.

In any case this limitation should be well understood when discussing this procedure, and its potentially intricate usage pattern with COMM_FREE, COMM_DISCONNECT, WIN_FREE and friends.