polymec / polymec-dev

The development "branch" of the polymec HPC libraries.
Mozilla Public License 2.0
2 stars 2 forks source link

Implement reduction for exchangers. #336

Closed pbtoast closed 5 years ago

pbtoast commented 5 years ago

Currently, the exchanger class assumes (implicitly) that data received from other processes all originates from a single source, as far as any given index is concerned. And in normal exchanger usage, this is certainly the case.

But a broad class of communication patterns exists in which several processes might contribute values to a single entity. For example, in one staggered-grid formulation of hydrodynamics, cell-centered values of mass and momentum are gathered to nodes and volume-averaged in order to construct nodal velocities. Specifically, each node gathers a portion of the mass and momentum from its surrounding cells and then decides what its velocity is.

There are other examples for faces and edges. If the exchanger class had a mechanism to detect cases in which aggregation/reduction takes place, and could select an operator to do the aggregation, such as

exchanger_set_reducer(ex, EXCHANGER_SUM)

or

exchanger_set_reducer(ex, EXCHANGER_MINPROC)

then the exchanger could support this class of communication patterns with no other modification. This is a big win if it can be made to work.

The operators must deal with integer, byte, and real- and complex-valued data, so we can't get crazy. Operators to consider: EXCHANGER_SUM - values are summed EXCHANGER_PRODUCT - values are multiplied EXCHANGER_MIN - minimum value wins EXCHANGER_MAX - maximum value wins EXCHANGER_MINPROC - value from lowest rank wins EXCHANGER_MAXPROC - value from highest rank wins