LLNL / conduit

Simplified Data Exchange for HPC Simulations
https://software.llnl.gov/conduit/
Other
212 stars 66 forks source link

add support for mpi-based `generate_*` functions #682

Closed xjrc closed 3 years ago

xjrc commented 3 years ago

In order to simplify the user burden for calling any of the blueprint::mesh::topology::unstructured::generate_* functions on a multi-domain mesh, the Blueprint library should include multi-domain/MPI extensions for these functions, which would handle the non-trivial task of generating new adjsets entries based on the function invocation and original mesh contents. Here's a rough sketch of the new functions to be introduced:

The sketch prototypes above deviate from their uni-domain counterparts in a few key ways, which are highlighted and explained below:

These interfaces/stipulations will be fleshed out more as their usage within real applications is further explored/manifested.

xjrc commented 3 years ago

After pondering this a bit more, I've come to the conclusion that the MPI-based generate_* functions should be centered around an adjset input rather than topology input. The primary reason for this is that using an adjset as input indicates that there are shared entities between domains that need to be coordinated, where a topology input leaves this as an open question (as well as which, if any, associated adjset entries need to be considered). With this being said, I propose the following alternative interfaces for these functions:

These new interfaces come with the following important considerations/caveats:

cyrush commented 3 years ago

@xjrc I agree this is a better path. Q: If we have a case where adjsets aren't in play (even in mpi land) -- folks can just use the non mpi variants? Or are their still some aspects of the MPI case that would be helpful to provide?

xjrc commented 3 years ago

@cyrush: I think the answer will be better illuminated after I write some code. My current feeling is that there will be some useful pieces of independent machinery in the MPI code (e.g. for each domain in a mesh, apply a transform to a specific topology and place the results in a uniform path), but I've not thought very hard about how these pieces might look as dedicated interfaces. We should revisit this after we can look at some semi-complete code.

cyrush commented 3 years ago

Sounds good

xjrc commented 3 years ago

After some more pondering, I've come to the conclusion that these functions and those being written by @nselliott in #642 will require some interface retooling. This is because, unlike other blueprint::mesh::*::* interfaces that can be called error-free assuming that the inputs pass a blueprint::mesh::*::verifycheck, the new blueprint::mpi::mesh::generate_* functions have a number of additional baked-in assumptions (e.g. that the given topologies and adjacency sets exist, that the given topology is unstructured, that the topologies match up across domains/ranks, etc.).

This could be helped by the following changes:

  1. Change the interface to be more like verify, e.g. bool blueprint::mpi::mesh::generate_*(..., Node &info) and allow for failure.
  2. Move the functions to some form of "unsafe" namespace to indicate that they aren't guaranteed to work on all inputs, e.g. blueprint::mpi::mesh::unsafe::generate_*.

@cyrush, @nselliott: Do you have any thoughts on this?

cyrush commented 3 years ago

I think they can check the preconditions and throw a descriptive error if those preconditions aren't met. Yes it's a shift from past, but I think there are other cases where we will hit this as well (given that an empty node is now valid)

cyrush commented 3 years ago

resolved in #767