Open gustavo-marques opened 1 year ago
I have a few questions for @alperaltuntas about this PE layout, that may lead to a few other updates...
ROOTPE_OCN=324
while keeping NTASKS_WAV=300
will let MOM6 have 24 nodes to itself instead of sharing one node with WW3ROOTPE_WAV=108
and then set ROOTPE_OCN
to an appropriate value to stay off the WW3 nodes?I think a handful of load balancing tests might lead to a more efficient layout.
@gustavo-marques when your run with 864 tasks for MOM6 finishes, can you post the timing summary?
We have two main wave grids we are working with: ww3a (3-degree grid) and wt0.66v1 (0.66 degree grid). With the 2-degree grid, WW3 cost is insignificant, and we can probably set NTASKS_WAV to 108 in that case. As for the wt0.66v1 grid, below are my comments.
NTASKS_WAV=300 is dictated by the way parallelism works in WW3, which is based on decomposing the spectral domain (24 by 25 = 600). Changing NTASKS_WAV from 300 to 288 or 324 would probably slow down WW3 enough to justify 12 idle cores (though this should be confirmed.).
CICE is sharing tasks 0-107 with the coupler and data models. Is it okay to dump WW3 on those tasks as well, or would it make more sense to set ROOTPE_WAV=108 and then set ROOTPE_OCN to an appropriate value to stay off the WW3 nodes
I strongly agree.
Changes ntasks_ocn from 300 to 864.