omlins / ParallelStencil.jl

Package for writing high-level code for parallel high-performance stencil computations that can be deployed on both GPUs and CPUs
BSD 3-Clause "New" or "Revised" License
311 stars 31 forks source link

Non cartesian gather! #92

Closed LaurentPlagne closed 1 year ago

LaurentPlagne commented 1 year ago

Hi,

I have an array quantity a computed on every process and I need to gather all the aon the process 0. Can I do this gathering operation with ParallelStencil.jl or should I switch to plain MPI programming ?

Thank you for your help.

luraess commented 1 year ago

Hi, gather! is an ImplicitGlobalGrid feature. You can achieve the gathering on rank 0 as shown in this example from the README. This will create a global array (in the example, for visualisation) that you can then handle serially on rank 0 if desired. If a custom gathering is needed, you could also provide it independently of the used packages using MPI functions.

LaurentPlagne commented 1 year ago

Thank you for your reply. In the example, the gather operation is done with Cartesian Arrays distributed on processes. In my case I have an array (local_history) defined on each process and I want to gather all local histories on process 0 (and make sum of all local_contributions).

I did use plain MPI:

       global_history_1D = MPI.Gather(local_history,0,comm)
       if me==0
            global_history = reshape(global_history_1D,(nh,nprocs))
            for h ∈ 1:nh
                psum = sum(global_history[h,:])
                scs.pressures[h]=psum
            end
        end

and it seems to give the correct result.