parthenon-hpc-lab / parthenon

Parthenon AMR infrastructure
https://parthenon-hpc-lab.github.io/parthenon/
Other
112 stars 35 forks source link

Integer-based swarm neighbor computation #1160

Open brryan opened 2 months ago

brryan commented 2 months ago

We moved to a floating point scheme to test which neighbors are adjacent to a given meshblock for swarm communication patterns. @lroberts36 pointed out that this can probably also be done with tree-based integer logic. We don't do that currently but his example code is preserved here if we want to move to this in the future.

   auto ll_block = pmb->loc.GetDaughter(0, 0, 0);
   int finest_level = pmb->loc.level() + 1;
   for (auto &n : pmb->neighbors) {
    std::vector<LogicalLocation> dlocs;
    auto &nloc =
        n.loc; // Would need to use the location in the coordinates of the origin tree
    if (nloc.level() == finest_level) {
      dlocs.emplace_back(nloc);
    } else if (nloc.level() == finest_level) {
      dlocs = nloc.GetDaughters(ndim);
    } else if (nloc.level() == finest_level - 2) {
      auto tlocs = nloc.GetDaughters(ndim);
      for (auto &t : tlocs) {
        auto gdlocs = t.GetDaughters(ndim);
        dlocs.insert(dlocs.end(), gdlocs.begin(), gdlocs.end());
      }
    } else {
      PARTHENON_FAIL("Proper nesting is not being respected.");
    }
    for (auto &d : dlocs) {
      const int k = d.lx3() - ll_block.lx3() + 1;
      const int j = d.lx2() - ll_block.lx2() + 1;
      const int i = d.lx1() - ll_block.lx1() + 1;
      if (i >= 0 && i <= 3 && j >= 0 && j <= 3 && k >= 0 && k <= 3)
        neighbor_indices_h(k, j, i) = n.gid;
    }
  }
lroberts36 commented 2 months ago

IIRC @brryan, this code didn't work as written?

brryan commented 2 months ago

@lroberts36 Yes I hit a PARTHENON_FAIL("Proper nesting is not being respected."); when trying this code snippet that I didn't understand. Reproducing your reply from #1090:

Yeah, as written it won't work for multi-tree forests or periodic boundaries. It is a one word change to make it correct when the neighbor logical location in the coordinates of the origin block is available though, I think.

but I didn't pursue this.

lroberts36 commented 2 months ago

haha, if I had only documented what the magical "one word change" was...