jsitaraman / tioga

Tioga is a library for overset grid assembly on parallel distributed systems
GNU Lesser General Public License v3.0
64 stars 36 forks source link

Tioga does not work reliably for 2D cases #44

Closed mennodeij closed 3 years ago

mennodeij commented 3 years ago

We run 2D cases as a single layer of cells in an array in X- and Y-directions, with the Z-coordinates at -1 and 1 to get a 2D-array of hexahedra. When using this method in an overset assembly (with both grids having nodes at Z=1 and Z=-1) TIOGA does not work reliably, i.e. there are extra fringe cells at the edges of the foreground grid and some fringe cells. Is this a known problem?

image image

mennodeij commented 3 years ago

correction: Z-coordinates are at 0 and 0.1

sayerhs commented 3 years ago

What is the second figure in showing? i.e., what are red, blue, and gray cells? Are they IBLANK values?

mennodeij commented 3 years ago

Thank you for your quick response and I’m sorry, that could have been more clear. Yes, the colors are iblank values: red is 1, white is 0 and blue is -1.

sayerhs commented 3 years ago

Thanks. I assume you're using the fully unstructured interface. In that case, it is expecting that the degrees of freedom are at the nodes and not cell-centers. For the inner mesh (foreground grid), it is not surprising that there are extra fringe cells when the mesh is rotated as it tries to provide a buffer between the receptor points on the two meshes. The pattern in the foreground mesh looks OK to me. The background mesh should not have any IBLANK=0 values. So that is definitely suspect. This is the iblank array that TIOGA returns, and you're interpreting it as existing on the nodes?

jsitaraman commented 3 years ago

Hello Menno, In the case you are simulating, can you confirm the following:

  1. For the background mesh both obcnode and wbcnode are nullsets
  2. For the foreground mesh obcnode has the list of nodal indices of all the hex cell faces that are on the perimeter (in the xy plane) The wbcnode is still a null set.

I haven't tested 2D type meshes in a while. I presumed it would work. The poisson_mms.f90 in driver folder does implement a 2-D type problem, but it uses grids in the X-Z plane and uses 2-cells (3-nodes) in the Y-direction.

thanks Jay

On Mon, Oct 19, 2020 at 10:50 AM Shreyas Ananthan notifications@github.com wrote:

Thanks. I assume you're using the fully unstructured interface. In that case, it is expecting that the degrees of freedom are at the nodes and not cell-centers. For the inner mesh (foreground grid), it is not surprising that there are extra fringe cells when the mesh is rotated as it tries to provide a buffer between the receptor points on the two meshes. The pattern in the foreground mesh looks OK to me. The background mesh should not have any IBLANK=0 values. So that is definitely suspect. This is the iblank array that TIOGA returns, and you're interpreting it as existing on the nodes?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/jsitaraman/tioga/issues/44#issuecomment-712334599, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACH3YMC4UXFJRAO45AK22IDSLR37XANCNFSM4SWJD6FA .

mennodeij commented 3 years ago

I'm using the fully unstructured interface, with hexahedral cells only. I have also passed in cell-iblank arrays, one for each sub-grid. The values on those cell-centred arrays is what is displayed. I think I need to be (more) careful about node-centred vs. cell-centred IBLANK values; I did not look at the node-centred values yet.

I am passing in wbcnodes for the background grid where the simulation has no-slip boundary condition, and obcnodes for the foreground grid describing the perimeter. I'll try without the wbcnodes on the background grid tomorrow, and I'll also take a look at the poisson_mss case, thanks for pointing that out.

sayerhs commented 3 years ago

@jsitaraman The overset rotating cylinder nightly test is a 2D simulation. So the 2D mesh overset connectivity is being regularly tested.

@mennodeij Regardless of what boundary conditions you set for your background mesh in your solver, it should not have any obcnode or wbcnode entries.

mennodeij commented 3 years ago

I have - finally - pinned down the problem with this case. For a reproducible case, see my branch https://github.com/mennodeij/tioga/tree/grid2d_driver (I can make a pull request if needed); see the driver/grid2d.f90 file and run the executable.

It seems to be related to the size of cells on the foreground and the background grid. In the test case, the cells are all of exactly the same size, and that leads to a checkerboard pattern of IBLANK values. If the cells on the foreground grid are slightly larger, the IBLANK values are not determined at all, and if they are slightly smaller they are OK (slightly on the order of 1E-8 of the grid size). The same holds - in reverse - for the cells of the background grid.

Is this something you can confirm, a known limitation or the assumption that the cells of the background grid are always larger than of the foreground grid?

image The foreground grid

sayerhs commented 3 years ago

@mennodeij When there are multiple grids overlapping in a certain region of space, by default, TIOGA uses the cell/node resolutions as the metric to determine who should solve the PDEs and who should end up as receptors. The logic being that we want to solve the equations on the finest mesh. So yes, there is an inherent assumption that the the mesh you want to end up as IBLANK=1 has a smaller resolution than the mesh you want to be a receptor. In your case, when the meshes are of the exact cell sizes, there is no real preference for one mesh or the other... so TIOGA just picks one or the other.

With meshes that have the same resolutions, you need to provide some additional metric to tell TIOGA how you want it to prefer one mesh over the other. You can bypass the internal cell/node volume computation done by TIOGA and pass your own cell/node resolutions through the Tioga::setResolutions method. This way, you can explicitly provide your own criteria to mark cells that you want to end up as IBLANK=1 and -1.

mennodeij commented 3 years ago

Ok, thanks for your very quick answer. I think that solves this issue so I will close it.