Behavior of non-parent PEs entering team split routines

[davidozog]

Should we clarify the behavior of non-parent PEs that enter a team split routine, i.e when those PEs pass parent_team=SHMEM_TEAM_INVALID?

In other words, does this existing text (in shmem_team_split_strided and shmem_team_split_2d):

This routine must be called by all PEs in the parent team.

imply this routine must only be called by PEs in the parent team? My guess is no, PEs that are not in the parent can pass SHMEM_TEAM_INVALID. This semantic generated some confusion in an SOS unit test:

https://github.com/Sandia-OpenSHMEM/SOS/issues/969

and I'm thinking something like this (but better;) might help prevent confusion:

This routine must be called by all PEs in the parent team, and any other PE calling the routine must set _parentteam to SHMEM_TEAM_INVALID.

There's also a slight bit of confusion with this existing text:

If the new team cannot be created or an invalid PE triplet is provided, then _newteam will be assigned the value SHMEM_TEAM_INVALID and _shmem_team_splitstrided will return a nonzero value on all PEs in the parent team.

in combination with:

If parent_team compares equal to SHMEM_TEAM_INVALID, then no new team will be created and _newteam will be assigned the value SHMEM_TEAM_INVALID [...] Return Values Zero on successful creation of new_team; otherwise, nonzero.

So it seems to be implied that PEs passing SHMEM_TEAM_INVALID should also return non-zero, but this may not be explicitly stated, unless I'm missing something. I might suggest:

If parent_team compares equal to SHMEM_TEAM_INVALID, then no new team will be created, _newteam will be assigned the value SHMEM_TEAM_INVALID, and the routine will return nonzero.

Sorry, lots of words to describe a pretty minor problem... :wink:

[jdinan]

You can have multiple simultaneous split operations, for example consider two splits where parents teams are even PEs and odd PEs. There is no semantic relationship between these splits even though they can be happening simultaneously. I think the same is true of a split involving SHMEM_TEAM_INVALID. There's no relationship to any other split that's happening. Given that it's impossible for a PE to have a handle for a team is does not belong to, I'm not sure there is anything to define for PEs outside of a given team. Whatever parent team argument they pass will define the requirements of that call.

Since a split on SHMEM_TEAM_INVALID does not create a team, our existing semantics imply that the return value should be nonzero.

[davidozog]

@jdinan - yeah... I retract my first suggestion then, thanks. ;)

For the 2nd suggestion, I agree it's implied that a split on the invalid team returns nonzero, I'm only saying it could be more explicit to avoid confusion. We can close this issue if you think it's clear though, I'm fine either way.

[jdinan]

@davidozog Always welcome suggestions to improve the standard.

[naveen-rn]

Not an item for errata - needs better wording.

[jdinan]

@davidozog I think we concluded that it's not broken, but the text could be improved. Is there anything more to do on this issue?

[davidozog]

Agreed and yes - I'll take a stab at improving it and post a PR asap.