Solver performance significantly degraded with AMR

[abarret]

The solver works great on uniform grids. With AMR, especially deep hierarchies or levels that have oddly shaped refined regions, the solver will sometimes stagnate. This doesn't appear to be a problem with overlapping patches.

My guess is our treatment of coarse-fine interfaces needs to be changed. This may be related to the discretization at CF interfaces being non SPD. If this is the case, I would expect dropping the time step size to result in better performance. Alternatively, we may be prolonging, restricting, and filling ghost cells incorrectly in the preconditioner at CF interfaces.

[abarret]

@bindi-nagda and I discussed this today. It seems that the solver works great with AMR as long as the refined region can be described as a single box and that CUBIC_COARSEN is not used. In particular, L-shaped domains provide poor preconditioner performance. The solver tends to stall with tolerance around 1e-7 or so.

There may be a bug in CUBIC_COARSEN, but it's not worth trying to track down right now; CONSERVATIVE_COARSEN should perform well enough.

The degradation may be related to how the corners at the bend in the L are treated in the smoothing routine. My intuition says that this would show up in the other IBAMR multigrid solvers, so there may be something that those do that we are not. The degradation could also be related to how CF interfaces are synchronized when we compute the residual.

[abarret]

The above is not completely true. It seems that the solver "works" for grids that have refined regions centered in the domain. However, I think that we are getting a benefit from some sort of symmetry in the problem, as the convergence properties of the solver are suboptimal (this doesn't make complete sense to me, but I have no other explanation). The residual tolerance is decreasing by a factor of 2-3 at each Krylov step with constant volume fraction and refined regions. On uniform grids, the residual tolerance decreases by roughly an order of magnitude with each Krylov step.

I'm still not sure what's happening here. I have a few ideas, but I'm not sure which are worth investigating.

• We synchronize the application of the operator, but we do not synchronize fluxes at coarse-fine interface.
• With variable volume fraction, are the discrete divergence and discrete gradient operators adjoints? This should be true for constant volume fraction, so it's unclear that this is causing an issue.
• Are we filling ghost cells at CF interfaces correctly in the smoother? F cycles do not work in IBAMR, presumably because the coarse and fine interfaces are communicating "incorrectly" (whatever that may mean). It's possible that with our anistropic problem, this becomes more important.
• Can we even solve the Poisson problem here? If we remove the divergence free condition and neglect pressure, does the solver work? This should be solvable: IBAMR already has variable coefficient Poisson solvers. (@bindi-nagda may be investigating this)
• There may be ideas we can use in the literature for variable coefficient single momentum NS solvers. I would be surprised if there are no multigrid solvers for these problems.

[bindi-nagda]

* Can we even solve the Poisson problem here? If we remove the divergence free condition and neglect pressure, does the solver work? This should be solvable: IBAMR already has variable coefficient Poisson solvers. (@bindi-nagda may be investigating this)

Without synchronization of fluxes at coarse-fine interfaces, implementing the above did not fix the issue with AMR grids. This test was done before PR #44.

[abarret]

Without synchronization of fluxes at coarse-fine interfaces, implementing the above did not fix the issue with AMR grids. This test was done before PR #44.

To clarify, you solved the coupled Poisson system (i.e. ignoring pressure and incompressibility), and you could solve the system if the synchronization in #44 occurre?.

[bindi-nagda]

Without synchronization of fluxes at coarse-fine interfaces, implementing the above did not fix the issue with AMR grids. This test was done before PR #44.

To clarify, you solved the coupled Poisson system (i.e. ignoring pressure and incompressibility), and you could solve the system if the synchronization in #44 occurre?.

Yes, I solved the system by ignoring pressure and incompressibility. However, I didn't solve the Poisson system using code from #44. I used the code prior to that PR. I could solve the system with a uniform grid, but the solver was stalling with refined regions i.e., displaying the same behavior as the full system before synchronization in #44 was implemented.

[abarret]

I think that has been mostly addressed and remaining open questions appear in #46.