The Earth System Modeling Framework (ESMF) is a suite of software tools for developing high-performance, multi-component Earth science modeling applications.
Currently, when doing I/O on Arrays and Fields where the distribution involves something other than 1 DE per PET, we do a redistribution to a default DistGrid (with 1 DE per PET) before doing the I/O. In discussions with the ESMF core team, we feel this may be unnecessary: since PIO does its own redistribution, it may be possible to avoid this extra initial redistribution step.
The easiest case to address here may be the one where we have 0 DEs on a PET: it seems like this case may already be handled, connected to my multi-tile work (where, for a given tile, some PETs won't have any DEs on that tile). So this part may be as easy as changing the redist_check function so it doesn't return true if there are PETs with 0 DEs, and then adding a unit test of this situation (and maybe also comparing the output before and after making this change to confirm that the output is the same in both cases).
Allowing > 1 DE per PET may require more changes, such as adding some loops in some places... although it's possible that it would already Just Work, so we should do some initial testing to see the current behavior.
This would improve performance and remove the current limitation that there need to be at least as many PETs as tiles in the multi-tile I/O case.
See also some slightly-related notes in #184 . In particular, if we allowed 0 DEs on a PET, then we could avoid the issue of having a DE with 0 elements.
Currently, when doing I/O on Arrays and Fields where the distribution involves something other than 1 DE per PET, we do a redistribution to a default DistGrid (with 1 DE per PET) before doing the I/O. In discussions with the ESMF core team, we feel this may be unnecessary: since PIO does its own redistribution, it may be possible to avoid this extra initial redistribution step.
The easiest case to address here may be the one where we have 0 DEs on a PET: it seems like this case may already be handled, connected to my multi-tile work (where, for a given tile, some PETs won't have any DEs on that tile). So this part may be as easy as changing the
redist_checkfunction so it doesn't return true if there are PETs with 0 DEs, and then adding a unit test of this situation (and maybe also comparing the output before and after making this change to confirm that the output is the same in both cases).Allowing > 1 DE per PET may require more changes, such as adding some loops in some places... although it's possible that it would already Just Work, so we should do some initial testing to see the current behavior.
This would improve performance and remove the current limitation that there need to be at least as many PETs as tiles in the multi-tile I/O case.
See also some slightly-related notes in #184 . In particular, if we allowed 0 DEs on a PET, then we could avoid the issue of having a DE with 0 elements.