Comparing Ghosting Functors

[permcody]

Per our discussion this morning. We talked about avoiding having duplicate relationship managers. In order for this to be a robust capability, we'll need to be able to compare the capabilities of different ghosting functor instances. I made an early attempt at doing this by requiring (at least in cases where there are multiple active RelationshipManagers) to compare against each other to see whether they should be appended to the existing list: https://github.com/idaholab/moose/blob/devel/framework/include/relationshipmanagers/RelationshipManager.h#L56-L59

This however, doesn't work when unrelated ghosting functors are compared even if one is a subordinate of the other. We need to come up with a more robust scheme to minimize the number of ghosting functors that have to be processed in a simulation.

The idea in MOOSE is that every object that requires "off element" information will be required to register a RelationshipManger so that we get the proper and minimum amount of ghosting in a simulation.

[roystgnr]

Throwing out ideas:

The idea of "traits" (in my head right now, a string->Real map under the hood) seems flexible. E.g. a side ghosting functor with n_levels=3 might ghost "side levels"->3; a similar connected ghosting ("point ghosting", "touch ghosting", whatever we call it) functor with n_levels=4 would ghost "connected levels"->4 and "side levels"->4 because the former entails the latter. Thus every map entry in the side functor would have a larger-valued entry in the connected functor, and the latter would supercede the former. Bump the latter down to n_levels<=2 and they would become incomparable, because for "connected levels" -inf<2 and for "side levels" 3>2.

Possible problems:

1. Not every partial ordering can be embedded in the extended real numbers. As an instructive example, my first failed idea was to use an integer (fine for things like n_levels, not so fine for things like "n meters away"). Are there going to be cases (e.g. subdomain-dependent ghosting levels) that would justify generalizing this much further, or allowing users to override it entirely?

2. If we invent new traits which subordinate old traits then this works fine (just give the new trait a new string) but if we invent new traits which are subordinate to old traits then we can't automatically take advantage of that without modifying all the old functors. Since this is just a performance rather than correctness problem I'm tempted to say "if you want stuff to go faster then modify your old functors" on the very rare occasions such a case crops up.

[roystgnr]

We do want operator<, not operator==, right? The latter can strip out conceptually duplicate functors but we'd also like to strip out conceptually subordinate functors.

[roystgnr]

Hmm... this stuff is only going to be called when ghosting functors are added or removed, like O(1) not O(Nelem) or O(Nproc) or anything, so there's no point in not making it more extensible. We can have the map point to vector<Real> and then if everything returns a one-entry vector until someday the need for per-subdomain-ghosting comes up, that's okay.

[permcody]

Yes - we want to use operator<. I wasn't being that ambitious when I started this a week ago.

[roystgnr]

No problem. My thinking on this is also rapidly evolving. For instance over the course of the afternoon I've progressed from "this belongs at the libMesh level because it's not MOOSE-specific and other users will see performance gains" to "be honest: this belongs at the libMesh level because it's not MOOSE-specific and other users will see performance gains and I want to butt in on every aspect of the design."