fmihpc / dccrg

Distributed cartesian cell-refinable grid
GNU Lesser General Public License v3.0
10 stars 10 forks source link

Change neighbor definition to "n unique cells in this direction" #38

Open ursg opened 8 months ago

ursg commented 8 months ago

This PR changes the fundamental definition of what a "neighbor" is. So far, in DCCRG, neighbor locations were always specified in units of the origin cell's size. At refinement level interfaces, this sometimes caused unintuitive behaviour, such as the same cell being addressed with different indices, or more cells than necessary being returned. In Vlasiator, this required some extensive workarounds, and a larger communication halo size (n=3) than would otherwise be required by the solvers (n=2).

With this PR, the definition of a neighbor at offset (x,y,z) is changed to mean "a cell that is encountered when walking along the highest refinement level, as the z-th unique cell in dimension 2, x-th cell in dimension 0 and y-th cell in dimension 1". This nontrivial dimension ordering, of course, stems from the internal use of vlasiator's translation solver.

This obviates most of the workarounds in the Vlasiator codebase, but probably breaks general-purpose usage of DCCRG, hence this PR is targeted towards the vlasiator-version branch. As a prototype implementation, no care has been taken to retain the previous structure, and a thorough refactoring would be required if both are to be used side-by-side.

This has been verified with Vlasiator global Magnetospheric runs.

iljah commented 8 months ago

Does this consider periodic grids? In that case one cell can be on several sides of another...

Naturally there are many preferred ways to build neighbor lists. In above case for example in periodic grid of two cells they can have only one neighbor but building any kind of solver on top of that doesn't seem easy. On other hand I can see that having same cell 10 times in one neighbor list could also be difficult to work with. Ideally users could choose how neighbor lists are built, maybe that would even be possible without too much effort.

ursg commented 8 months ago

Yes, grid periodicity is respected in the same way as before. For directions that are single-cell-periodic (such as Vlasiator 2D runs), an invalid cell ID is returned for any neighbor in that collapsed dimension.

markusbattarbee commented 1 month ago

Noting that due to the method currently actually going through all 6 permutations of search directions, a single cell can again be found at several different offset values.