Adaptive mesh refinement in specific region of computational domain

[mameehan5]

Hi all,

Is there any interest in combining the static and adaptive mesh refinement approaches? I am finding many instances where I would like to adaptively refine only a specific region of the flow with a certain criteria. It would look something like:

amr.gradrho.max_level = 2
amr.gradrho.adjacent_difference_greater = 0.01
amr.gradrho.field_name = density
amr.gradrho.in_box_lo = -1.0 -1.0 0.0
amr.gradrho.in_box_hi =  1.0  1.0 2.5

This would only tag density differences within the specified in_box region.

Thanks! Mike

[drummerdoc]

Hey Mike, seems like the text and the suggested inputs are not aligned, in that your inputs could be adaptive if the density is changing over time. If you want static refinement in regions, there are couple ways to do it, but based on your inputs above, you could simply refine within a subbox when the density is greater than zero (which is always, hopefully). If this is your only criteria, you could set the refinement interval to be 1000000 or something so that it never checks again. But combining a criteria that never changes which points it tags with another that is dynamic is not an issue. Perhaps I'm not understanding what you want though. Let me know

[mameehan5]

Thanks for the reply Marc! Sorry I was not clear. I was thinking of a feature that allows dynamic refinement but only to a subset of the computational domain. In my example, if the computational domain was, say

geometry.prob_lo     = -1.5 -1.5 0.0
geometry.prob_hi     =  1.5  1.5 3.0

the inputs provided above would adaptively refine density variations within the specified region, leaving the outer bounds of the computational domain unresolved. Is there a way to do this with the current implementation? This is not necessarily an issue as opposed to a feature.

[drummerdoc]

Your usage seems correct. each refinement criteria takes optional arguments "in_box_lo" and "in_box_hi" that define the physical coordinates of where that criteria will be applied. It should not tag any points outside that box (except by way of the "n_error_buf" functionality, and this box defaults to the entire domain. Let me know if you see behavior suggesting that it doesn't work as expected.

[mameehan5]

Yep, that is exactly how I was thinking this would work. But, the currently implementation of LMeX supports either AMR or specification of in_box, not simultaneously. I posted this issue as a question of if there was interest in this sort of behavior. I wrote a bit of hack way to do this a while ago, and if there was broader interest, I could flush it out and submit a PR. It would probably be written in a more sustainable way, however, if a current dev implemented this.

[drummerdoc]

I think I agree with you, but this is what I'd call a bug. Yes, for each specific refinement criteria, the tagging should work only on the intersection of each box with the m_domain_box at at that level associated with that criteria, and m_domain_box should default to the entire problem domain box. I'm not sure how it became what it is now, but I'm good with being able to set the refinement criteria box as a subdomain in physical coordinates of the entire domain, and being able to set a condition within that box that is always true (such as density > -1). IT would be great if you submitted a PR to AMReX and tag me as a reviewer. Thanks for pointing that out!

[mameehan5]

Okay, sounds good. I am going to be out of the office for the next few weeks, but I will put it on the todo list for when I get back!

[drummerdoc]

Excellent. In theory, it should be a lot faster if the subbox is a tiny fraction of the domain too (although it's rare that regrinding takes too much time anyway). thanks for pointing this out.

[mameehan5]

Coming back to this... I was able to get this feature to work, but it does not seem like AMReX supports in_box combined with other refinement method. To navigate around this, I added a few lines near the end of AMReX_ErrorList.cpp to unrefined outside the box

                // MAM: untag cells that are outside of the box
                const auto plo = geom.ProbLoArray();
                const auto dx  = geom.CellSizeArray();
                const auto tag_rb = m_info.m_realbox;
                ParallelFor(tba, [=] AMREX_GPU_DEVICE (int bi, int i, int j, int k) noexcept
                {
                    GpuArray<Real,AMREX_SPACEDIM> pt
                        {AMREX_D_DECL(plo[0]+(Real(i)+Real(0.5))*dx[0],
                                      plo[1]+(Real(j)+Real(0.5))*dx[1],
                                      plo[2]+(Real(k)+Real(0.5))*dx[2])};
                    if (!tag_rb.contains(pt.data())) {
                        tagma[bi](i,j,k) = clearval;
                    }
                });

and on the PeleLMeX side, I set the box size to the domain is in_box is not present. This can be seen in my branch refine_in_box branch on my fork.

In my opinion, this is a bit of a hack so I have not initiated a PR. If this is something we want to pursue further, we will probably need support on the AMReX side to allow something like this. I wasn't sure how to do it cleanly/safely, but it should be possible with a little more thought.

[drummerdoc]

I'd prefer to tag cells you want rather than untagging those you don't. The issue is getting "proper nesting" correct. It turns out to be really complicated figuring out that bit. We had something like that for preventing c-f boundaries at outflows and it was a mess. I think that if you made every refinement criteria using the "in_box" argument it should do what you want, no?