search

JuliaParallel / Elemental.jl

Julia interface to the Elemental linear algebra library.
Other
78 stars 15 forks source link

Examples of Communcations Patterns in Elemental #73

Open JBlaschke opened 3 years ago

JBlaschke commented 3 years ago

I am not sure how to convert a local Array into Elemental.DistMatrix. The example in the README (and that I have used in #72 ) use Elemental.gaussian to fill a DistMatrix. However for a realistic workflow, I need to load distributed pieces of data (each local to a rank) and "fill" the DistMatrix locally. I seem grid objects mentioned, so there must be some way to specify which indices map to which ranks (like https://github.com/eth-cscs/ImplicitGlobalGrid.jl), but I cannot find any documentations/examples

Clearly Elemental.gaussian! operates locally for each rank -- e.g. if I do:

rank = MPI.Comm_rank(comm)
A = Elemental.DistMatrix(Float64);
if rank == 0
Elemental.gaussian!(A, 4000, 3200);
end

the program hangs.

Can you point me to the right place?

andreasnoack commented 3 years ago

Take a look at https://github.com/JuliaParallel/Elemental.jl/blob/83089155659739fea1aae476c6fd492b1ee20850/test/lav.jl#L39-L74. This is a distributed sparse matrix but the process is similar in the dense case. You first select all the elements to be distributed from local arrays with queueUpdate and then you process the queries.