search

cu-hpsc / hpsc-hw3

BSD 2-Clause "Simplified" License
0 stars 0 forks source link

DmLocalToLocalBegin/End #5

Open mwatwood-cu opened 5 years ago

mwatwood-cu commented 5 years ago

I'm working on getting the Gauss-Seidel running in place and I have the forward substitution running and I am trying to get the communication working appropriately. The communication code below works and provides convergence. However the readme says to use the DMLocalToLocalBegin and End methods where I am using DMLocalToGlobal. 

  ierr = DMGetLocalVector(dm, &Xlocal);CHKERRQ(ierr); // A work vector
  ierr = DMGlobalToLocal(dm, U, INSERT_VALUES, Xlocal);CHKERRQ(ierr);
  ierr = DMDAGetLocalInfo(dm,&info);CHKERRQ(ierr);
  ierr = DMDAVecGetArray(dm, Xlocal, &u);CHKERRQ(ierr);
  ierr = DMDAVecGetArray(dm, V, &v);CHKERRQ(ierr);

  ...

  ierr = DMDAVecRestoreArray(dm, Xlocal, &u);CHKERRQ(ierr);
  ierr = DMDAVecRestoreArray(dm, V, &v);CHKERRQ(ierr);
  ierr = DMLocalToGlobal(dm, Xlocal, INSERT_VALUES, U);CHKERRQ(ierr);

First is it required to use the LocalToLocal?

If so, then am I thinking about this correctly? What I have is overkill and I don't need to push back my entire u matrix into global U each time as only the ghosted edges are needed for an update in the next step between MPI processes.

However, if I do this I either get no update at all (where my updated calculations get overwritten) or I get a corrupted memory error.

So my questions are: Is my problem in how I'm reading at the start of my residual rather than how I'm finishing it perhaps? Should the LocalToLocalBegin come before my calculations and the End come after or should they both come after the computations are finished? Lastly, using LocalToLocal should my two vectors I'm using be U and Xlocal or Xlocal twice?

jedbrown commented 5 years ago

It is not required (see also #1), just an option that can do the required communication in one step instead of two (global-to-local followed by local-to-global), albeit at some extra coding complexity. The other questions are more subjective (there is no one right way to do it) and I'll leave it to other students to discuss the merits of any particular choice.