openshmem-org / specification

OpenSHMEM Application Programming Interface
http://www.openshmem.org
50 stars 38 forks source link

Clarify SHMEM_TEAM_SHARED membership via shmem_ptr #325

Closed davidozog closed 4 years ago

davidozog commented 4 years ago

This ticket reverts the proposed change from #307 (to relax the correspondence of SHMEM_TEAM_SHARED with shmem_ptr) and instead specifies that all symmetric heap objects on the PEs must be accessible for those PEs to be included in SHMEM_TEAM_SHARED.

Some more background information: We learned from #307 that we generally prefer SHMEM_TEAM_SHARED and shmem_ptr to have a direct correspondence. We also realized that the shmem_ptr interface allows implementations to freely limit accessibility, which resolves any problems with representing non-triplet teams (thanks @minsii!). We also became aware that shmem_ptr may lack "symmetry" with regard to other PE's, and may lack "coverage" with regard to the whole symmetric heap/data segments (discussed here). This PR (#325) addresses the symmetry/coverage issues of shmem_ptr by proposing that the entire symmetric heap be accessible for SHMEM_TEAM_SHARED membership. (It does not prohibit membership if objects on the data segment are also accessible).

Side note: Another option may be for SHMEM_TEAM_SHARED membership to require PE accessibility across both the symmetric heap and data segments (but apparently some implementations may not support shmem_ptr on the data segment). In that case, it may be cleaner to rely on the shmem_pe_accessible routine rather than "shmem_ptr with caveats" to define SHMEM_TEAM_SHARED...

minsii commented 4 years ago

The clarification makes sense to me :+1:

Besides, I think the option that relies on shmem_pe_accessible might not work, because shmem_pe_accessible checks whether the PE can be accessed by using SHMEM operations (e.g., the PE can be on a remote node).