search

mpiwg-rma / rma-issues

Repository to discuss internal RMA working group issues
1 stars 0 forks source link

Calling MPI_Request_free on RMA requests #29

Open devreal opened 2 months ago

devreal commented 2 months ago

Does anyone remember why we explicitly disallow MPI_Request_free on RMA requests? If the window has been flushed we might as well release the requests via MPI_Request_free in addition to test & wait.

We have an errata that roles back a change we made accidentally in 4.1 that allowed this behavior was contradicting language explicitly disallowing the use of MPI_Request_free: https://github.com/mpi-forum/mpi-standard/pull/904

The language in question is here: https://github.com/mpi-forum/mpi-standard/blob/6d3fc03aac92c0ac62dcb7638bb0a8935aaa2081/chap-one-side/one-side-2.tex#L2024

jeffhammond commented 2 months ago

Unlike Send, there's no way to know RMA completed otherwise. There's no valid use case for allowing it and plenty of harm.

devreal commented 2 months ago

You know that the operation is complete once MPI_Win_flush returns. After that, all that is left to do is to release the request back to MPI.

jeffhammond commented 2 months ago

Fine but that's pointless. Don't use request based RMA if you're going to complete with flush.

jeffhammond commented 2 months ago

The use case for freeing RMA requests is incompatible with the use case for getting RMA requests in the first place.

jdinan commented 2 months ago

You might need to mark outstanding RMA request objects as completed during the flush in order to support this. One design principle the Forum has tried to be consistent about is that MPI should not require implementations to internally track request objects that are returned to the user. But, we would need to know that the communication was completed in order to make these objects safe to free. As an alternative, you could track epoch ID on the window and store the epoch ID in the request object to infer that its epoch was already cleared by a flush. You would need to do this per destination for MPI_Win_flush vs MPI_Win_flush_all, but it's probably doable. If the epoch ID counter rolls over, you may need to fall back to a flush in MPI_Request_free, which could be disallowed since this operation is required to be local.

However, I do agree with Jeff's point that this doesn't feel like a valid RMA use case and it will create additional overheads from bookkeeping request status.

~Jim.

On Mon, Sep 23, 2024 at 2:18 AM Jeff Hammond @.***> wrote:

The use case for freeing RMA requests is incompatible with the use case for getting RMA requests in the first place.

— Reply to this email directly, view it on GitHub https://github.com/mpiwg-rma/rma-issues/issues/29#issuecomment-2367314212, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZ5P4XYEEQTSBAGFDP6PATZX6XDBAVCNFSM6AAAAABOVCYNVSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRXGMYTIMRRGI . You are receiving this because you are subscribed to this thread.Message ID: @.***>