New assertions for passive-target epochs

[mpiforumbot]

Originally by jhammond on 2013-10-22 10:20:31 -0500

---

The new RMA synchronization mode(s) provided by MPI-3 motivate new assertions.

How to read this ticket

Each of these assertions should be considered as a separate proposal. They are included together here to keep things simple.

Proposed Assertions

-MPI_MODE_NOSTORE* -- This assertion is already defined and used for active-target synchronization but should now exist for passive target as well.

-MPI_MODE_NOLOAD* -- Identical to _MPI_MODENOSTORE except for load instructions.

-MPI_MODE_NOSYNC* -- The application will not call _WINSYNC within the epoch.

-MPI_MODE_NOFLUSH* -- The application will only use the passive target synchronization style of MPI-2 RMA, i.e. no variant of flush will be called between _WIN_LOCK(ALL) and _WIN_UNLOCK(ALL).

-MPI_MODE_NOFLUSHALL* -- The application will not use _MPI_WIN_FLUSH(_LOCAL)ALL during a passive target epoch.

-MPI_MODE_NOREATTACH* -- The application will not call _MPI_WINATTACH and _MPI_WINDETACH on the same buffer during a passive target epoch.

Proposed Text

Page 452 lines 22-25 will be augmented by the following bold face text:

MPI_MODE_NOCHECK -- no other process holds, or will attempt to acquire, a conflicting lock, while the caller holds the window lock. This is useful when mutual exclusion is achieved by other means, but the coherence operations that may be attached to the lock and unlock calls are still required.

-_MPI_MODENOLOADSTORE -- the local window will not be updated by stores (or MPI calls) nor will it be accessed by loads (or by MPI calls) since last synchronization; only the public window will be updated or accessed by RMA communication operations. This assertion may allow the implementation to eliminate expensive memory synchronization.*

-_MPI_MODE_NOSYNC -- MPI_WINSYNC will not be called during this lock(all)-unlock(all) epoch. This assertion may allow the implementation to eliminate expensive memory synchronization.*

-_MPI_MODE_NOFLUSH -- MPI_WIN_FLUSH and MPI_WIN_FLUSHLOCAL will not be called during this lock(all)-unlock(all) epoch.*

-_MPI_MODE_NOFLUSHALL -- MPI_WIN_FLUSH_ALL and MPI_WIN_FLUSH_LOCALALL will not be called during this lock(all)-unlock(all) epoch. This assertion may allow the implementation to eliminate state associated with tracking the outstanding operations on the window.*

-_MPI_MODENOREATTACH -- Allows implementation to cache buffer registration more aggressively.*

Question: Is "local window" the right term? I thought we talked about public and private windows.

Motivation

When the application does not use load/store against a window, the implementation may be able to avoid memory barriers or other internal synchronization operations.

MPI_MODE_NOFLUSHALL may allow an implementation to avoid using state to track active target ranks. To optimize the sparse-rank-list case of FLUSH_ALL, the implementation might keep track of all the ranks that are targeted so it can flush them rather than flushing all ranks for the window (which is sufficient but not necessary). If the user intends to do this above MPI or otherwise does not need FLUSH_ALL, then the implementation can skip this state and always flush the remote rank in FLUSH. An implementation might use that active target list to make FLUSH a no-op when called unnecessarily (i.e. for a rank the is not the target of any RMA operation during an epoch).

MPI_MODE_NOSYNC may be useful for the SEPARATE memory model, because it tells the implementation that it does not have to consider the possibility that the public and private window will be synchronized during the epoch.

[mpiforumbot]

Originally by balaji on 2014-03-04 12:31:49 -0600

---

Yes (need to go and figure out the exact assertions): 6 No: 0

[mpiforumbot]

Originally by jhammond on 2014-12-09 11:16:42 -0600

---

Dec. 2014 WG discussion was favorable for all of this.

[mpiforumbot]

Originally by gropp on 2014-12-10 12:53:50 -0600

---

The working group request a complete proposal, with text as it would be added to the RMA chapter.

[mpiforumbot]

Originally by rsthakur on 2015-06-03 12:09:28 -0500

---

The proposal above says "Page 452 lines 22-25 will be augmented by the following bold face text". However, there is no bold face text.

[mpiforumbot]

Originally by jhammond on 2015-06-03 12:48:09 -0500

---

Yes, I noticed. I don't remember why it's there but haven't had time to resolve the diff to know what should be bold.

[mpiforumbot]

Originally by rsthakur on 2015-06-03 13:09:01 -0500

---

From the June 2015 Forum meeting: For each of the proposed asserts, some implementer needs to describe how they would take advantage of it.