How Adapting AmrCoreFill for CPU usage

[ztdepztdep]

Hi everyone,

I'm working on the example Amrcore, and i want set the Directlet bc . and i have defined the involves a boundary function AmrCoreFill and the values of the bcs were also changed. but it seems the code doesn't give and change.

However, it seems to be designed exclusively for GPU execution. I'm hoping to transfer its functionality to the CpuBndryFuncFab class for execution on the CPU.

Here's a relevant code snippet: if(Gpu::inLaunchRegion()) { GpuBndryFuncFab<AmrCoreFill> gpu_bndry_func(AmrCoreFill{}); } else { CpuBndryFuncFab bndry_func(nullptr); // Without EXT_DIR, we can pass a nullptr. }

What steps are necessary to adapt AmrCoreFill for compatibility with CpuBndryFuncFab? I'd appreciate any insights or guidance on this matter. Thank you!

[WeiqunZhang]

The way that the example was written was unfortunately confusing because it was written at a time people were transiting from existing CPU codes to GPU capable codes. It allowed CPU codes to be ported to GPU piece by piece.

You could remove all the if (GPU::inLaunchRegion()) tests, keep all the if blocks and remove all the else blocks. That would remove the need to use CpuBndryFuncFab.` The resulting code should be able to run on CPU if the code is built for CPU. The word GPU in GpuBndryFuncFab should be regarded as GPU ready if it's built for GPU, not GPU exclusive.

(If you are still curious, CpuBndryFuncFab expects a function pointer with the signature of https://github.com/AMReX-Codes/amrex/blob/47347f785f5c26f1f9e65bef8e10f2d383406ef7/Src/Base/AMReX_PhysBCFunct.H#L28)

[ztdepztdep]

> The way that the example was written was unfortunately confusing because it was written at a time people were transiting from existing CPU codes to GPU capable codes. It allowed CPU codes to be ported to GPU piece by piece.

You could remove all the if (GPU::inLaunchRegion()) tests, keep all the if blocks and remove all the else blocks. That would remove the need to use CpuBndryFuncFab.` The resulting code should be able to run on CPU if the code is built for CPU. The word GPU in GpuBndryFuncFab should be regarded as GPU ready if it's built for GPU, not GPU exclusive.

(If you are still curious, CpuBndryFuncFab expects a function pointer with the signature of

https://github.com/AMReX-Codes/amrex/blob/47347f785f5c26f1f9e65bef8e10f2d383406ef7/Src/Base/AMReX_PhysBCFunct.H#L28 )

I have done this. and I changed the bc to the following

int bc_lo[] = {BCType::ext_dir, BCType::foextrap, BCType::int_dir}; 
int bc_hi[] = {BCType::ext_dir, BCType::foextrap, BCType::int_dir};

and the BCFill.h are as following:

#ifndef BCFILL_H
#define BCFILL_H
#include <AMReX_FArrayBox.H>
#include <AMReX_Geometry.H>
#include <AMReX_PhysBCFunct.H>

struct AmrCoreFill
{
    AMREX_GPU_DEVICE
    void operator() (const amrex::IntVect& iv, amrex::Array4<amrex::Real> const& data,
                     const int dcomp, const int numcomp,
                     amrex::GeometryData const& geom, const amrex::Real time,
                     const amrex::BCRec* bcr, const int bcomp,
                     const int orig_comp) const
        {
            amrex::Print() << "\n We are running in this  " <<bcr[0]<<"  "<< bcr[1]<< std::endl;        
        const int ilo = geom.Domain().smallEnd(0);
        const int ihi = geom.Domain().bigEnd(0);

        const int jlo = geom.Domain().smallEnd(1);
        const int jhi = geom.Domain().bigEnd(1);

        const auto [i,j,k] = iv.dim3();

        if (i < ilo) {
            data(i,j,k) = 10.0;
        }
        if (j < jlo) {
            data(i,j,k) = 10.0;         
                 }
        }
};

#endif

the prompt I added inside the operator() function doesn't seem to be executed, and the program continues to run with the original periodic boundary conditions. am i missing someting ?

[WeiqunZhang]

If possible, show the output of git diff. If you have your own inputs file, show that too. More information is almost always better.

My guess is geom.is_periodic is still all 1 in the inputs file.

[WeiqunZhang]

If that is the case, you would want to set geom.is_periodic = 0 0 1 to be consistent with

int bc_lo[] = {BCType::ext_dir, BCType::foextrap, BCType::int_dir}; 
int bc_hi[] = {BCType::ext_dir, BCType::foextrap, BCType::int_dir};

That is only the last direction is set to be periodic (i.e., internal Dirichlet).

If that is not the case, we would need to know about the changes made.

[ztdepztdep]

great, it works.