GeorgeVandenberghe-NOAA
commented
3 years ago I finally have corners in the EXCH.f working. Corner exchange code looks like this
!! corner points. After the exchanges above, corner points are replicated in ! neighbour halos so we can get them from the neighbors rather than ! calculating more corner neighbor numbers ! A(ista-1,jsta-1) is in the ileft a(iend,jsta-1) location ! A(ista-1,jend+1) is in the ileft a(iend,jend+1) location ! A(iend+1,jsta-1) is in the iright a(ista,jsta-1) location ! A(iend+1,jend+1) is in the iright a(ista,jend+1) location ibl=max(ista-1,1) ibu=min(im,iend+1) jbu=min(jm,jend+1) jbl=max(jsta-1,1)
call mpi_sendrecv(a(iend,jbl ),1, MPI_REAL,iright,1 , &
& a(ibl ,jbl ),1, MPI_REAL,ileft ,1, &
& MPI_COMM_COMP,status,ierr)
call mpi_sendrecv(a(iend,jbu ),1, MPI_REAL,iright,1 , &
& a(ibl ,jbu ),1, MPI_REAL,ileft ,1, &
& MPI_COMM_COMP,status,ierr)
call mpi_sendrecv(a(ista,jbl ),1, MPI_REAL,ileft ,1, &
& a(ibu ,jbl ),1, MPI_REAL,iright,1, &
& MPI_COMM_COMP,status,ierr)
call mpi_sendrecv(a(ista,jbu ),1, MPI_REAL,ileft ,1 , &
& a(ibu ,jbu ),1, MPI_REAL,iright,1, &
& MPI_COMM_COMP,status,ierr)It does work. I need to strip out a lot of debug code before publishing I tested it by exchanging with an array of coordinates and ensuring coordinates matched I and J for all I, J from ista-1 to iend+1 and jsta-1 to jend+1 with the constraints these had to be in the bounds 1:im,1:jm A second possible test would to be to broadcast one full domain 2D array and then also scatter and exchange it and compare the exchanged results with the broadcast subcomain. These should be exact.
JesseMeng-NOAA
HuiyaChuang-NOAA
WenMeng-NOAA
hertneky
fossell
EMC_post is currently decmposed on latitude (J) only. This is adequate for several more years but since post is generally being refactored, now is a good time to make the jump to 2D. A second goal is to make the 2D decomposition either flexible, or just have it mimic the ufs-weather-model decomposition so developers working on both codes can exploit commonality. This will be a modestly difficult project with most effort, figuring out the plumbing of the code (in progress). This issue is being created for management and project leader tracking and per EMC management directives and also best practices, results should be tracked through this Github issue or slack, NOT email.
There are many OTHER scaling issues in the post that are not affected by the decomposition. Most of the issues are orthogonal to the decomposition though and can be worked independently. The most salient is input I/O of model state fields in the standalone post.
By 03/01/2021:
The offline post testing procedure provided by Jesse can be found at here
The inline post testing procedure provided by Bo can be found at here
Jesse's FV3 branch can be found at here