Brief explanation of the problem: when assembling an AffineFEOperator such that:
the linear form has terms that touch ghost DoFs which are not touched by the assembly of the terms in the bilinear form
GridapDistributed.jl wrongly computes the communication pattern which is required to properly assemble the right hand side vector of the linear system.
This is so because it (wrongly) assumes that the ghost layout of the PRange describing the partition of the rows of the linear system coefficient matrix matches that of the right hand side vector. This is general true, but not necessarily, as, e.g., in the mwe example below.
I will try to fix the issue in a PR following this issue.
using Gridap
# Parallel distributed packages
using PartitionedArrays
using GridapDistributed
using Test
function run_app(rank_partition,distribute)
DX = 1000.0
DY = 1000.0
order = 0
n_els_x = 4
n_els_y = 4
dx = DX/n_els_x
domain = (0,DX,0,DY)
cell_partition = (n_els_x,n_els_y)
ranks = distribute(LinearIndices((prod(rank_partition),)))
model = CartesianDiscreteModel(ranks,rank_partition,domain,cell_partition; isperiodic=(false,false))
Ω = Triangulation(model)
dΩ = Measure(Ω, 5*(order+1))
Γ = SkeletonTriangulation(model)
dΓ = Measure(Γ, 5*(order+1))
Q = FESpace(model, ReferenceFE(lagrangian, Float64, order), conformity=:L2)
P = TrialFESpace(Q)
# initial conditions
ph=FEFunction(P,prand(partition(Q.gids)))
b(q) = ∫(jump(ph)*mean(q))dΓ
m(p,q) = ∫(p*q)dΩ
op = AffineFEOperator(m, b, P, Q)
b=assemble_vector(b(get_fe_basis(Q)),P)
tol=1.0e-12
@test norm(op.op.vector-b)/norm(b) < tol
end
rank_partition = (2,1)
with_debug() do distribute
run_app(rank_partition,distribute)
end
The MWE below reproduces the error.
Brief explanation of the problem: when assembling an
AffineFEOperator
such that:the linear form has terms that touch ghost DoFs which are not touched by the assembly of the terms in the bilinear form
GridapDistributed.jl
wrongly computes the communication pattern which is required to properly assemble the right hand side vector of the linear system.This is so because it (wrongly) assumes that the ghost layout of the PRange describing the partition of the rows of the linear system coefficient matrix matches that of the right hand side vector. This is general true, but not necessarily, as, e.g., in the mwe example below.
I will try to fix the issue in a PR following this issue.