Closed hannesbrandt closed 2 weeks ago
The interface and options look good! I don't have any requests for changes.
Great, then I will turn this into a normal PR, mainly for the interface.
The skip_local
and skip_refine
options will follow in future PRs.
This PR adds a new interface for partitioning patch data in different modes, with one of them being already implemented.
We introduce a new workflow for partitioning patch data:
fclaw2d_domain_iterate_pack
. The function receives a user-definedfclaw2d_pack_callback_t
, which packs the data of a patch into a provided storage location. Then the function starts sending the packed data according to the new partition boundaries.fclaw2d_domain_iterate_transfer
. The function uses afclaw2d_transfer_callback_t
, which may be a simple copy or reassignment of the patch data.fclaw2d_domain_iterate_transfer
may be called beforeiterate_pack
or afteriterate_unpack
instead.fclaw2d_domain_iterate_pack
has to be succeeded by a call tofclaw2d_domain_iterate_unpack
. This function unpacks the repartitioned patch data it received by a user-definedfclaw2d_unpack_callback_t
.fclaw2d_domain_partition_free
frees all partition-specific context data.The workflow has to be called after
fclaw2d_domain_partition
, which repartitions the patches without the patch data and updates the partition boundaries as required. For now we keep the old workflow of usingassign_before_partition
,retrieve_after_partition
anditerate_partitioned
. It may be useful for writing a comparison paper on different partitioning modes.The implementation currently comes with a single mode that packs all patches into the send buffer, even if they are not sent after all. For future additional modes, we propose to add two partitioning options,
skip_local
andskip_refine
, which can be combined arbitrarily:skip_local
enabled, we pack data only for the patches that change process during repartitioning.skip_refine
enabled, we change the packing behavior for the last level of newly refined patches. To achieve this, we save the local patch indices of newly refined patches during the last call to{domain,wrap}_adapt
. Based on these indices, only the first child of a family of newly refined patches calls thefclaw2d_pack_callback_t
. Subsequently, thefclaw2d_unpack_callback_t
is called for all children with the packed data from the first child.Does this interface with the corresponding options seem convenient to you? We would appreciate your feedback/ideas on this topic!