remove copy of state into mpi window

hkershaw-brown commented 2 months ago

Description:

Puts the %copies array into the mpi window for get_state (forward operator) Removes the copy of the %copies(1:real_ens_members, :) to the window. This reduces the per-code memory usage (removes a %copies(1:real_ens_members, 1:my_num_vars) ~1/2 for the state per memory core usage.

The offset is calculated from %num_copies, the get_state is only the real ensemble members

ink

I think this is a good time to remove the cray window module.

Fixes issue

fixes #718

Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation update

Documentation changes needed?

[ ] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.

Tests

There is a developer test included. Note this needs fortran-testanything code to run. Bitwase lorenz_96 mpi, mpif08, nompi

Checklist for merging

[ ] Updated changelog entry
[ ] Documentation updated
[ ] Update conf.py

Checklist for release

[ ] Merge into main
[ ] Create release from the main branch with appropriate tag
[ ] Delete feature-branch

Testing Datasets

[ ] Dataset needed for testing available upon request
[ ] Dataset download instructions included
[ ] No dataset needed

hkershaw-brown commented 1 month ago

@hkershaw-brown check PMO

mjs2369 commented 3 weeks ago

Do we want to rename the window mods to not mention cray?

no_cray_win_mod.f90 -> win_mod.f90 no_cray_winf08_mod.f90 -> winf08_mod.f90

mjs2369 commented 3 weeks ago

In buildfunctions.sh and buildconvfunctions.sh, you can remove all mentions of windowsrc. It's not needed anymore.

I just pushed up my commits from when I removed the cray window a while back if you want to take a look at these or cherry pick and commits (on branch remove_craywin). It has windowsrc already taken out of the code. It also has the window files already renamed and updates buildfunctions.sh and buildconvfunctions.sh accordingly

hkershaw-brown commented 3 weeks ago

In buildfunctions.sh and buildconvfunctions.sh, you can remove all mentions of windowsrc. It's not needed anymore.

I just pushed up my commits from when I removed the cray window a while back if you want to take a look at these or cherry pick and commits (on branch remove_craywin). It has windowsrc already taken out of the code. It also has the window files already renamed and updates buildfunctions.sh and buildconvfunctions.sh accordingly

Good point. I shall grab your changes from remove_craywin. Since we only have null/mpi/mpif08 windows we don't have to have a separate window file (cray vs not cray). I quite like the window being in a separate file rather than adding more stuff to {null|mpi}_utilties_mod.f90

hkershaw-brown commented 3 weeks ago

In buildfunctions.sh and buildconvfunctions.sh, you can remove all mentions of windowsrc. It's not needed anymore.

I just pushed up my commits from when I removed the cray window a while back if you want to take a look at these or cherry pick and commits (on branch remove_craywin). It has windowsrc already taken out of the code. It also has the window files already renamed and updates buildfunctions.sh and buildconvfunctions.sh accordingly

cherry-picked your commits onto this branch

hkershaw-brown commented 3 weeks ago

Remove these identical comments -

https://github.com/NCAR/DART/blob/8c549735550e7120ee926f3b194ee8f0cba5ada7/assimilation_code/modules/utilities/no_cray_win_mod.f90#L32-L33

https://github.com/NCAR/DART/blob/8c549735550e7120ee926f3b194ee8f0cba5ada7/assimilation_code/modules/utilities/no_cray_winf08_mod.f90#L32-L33

done

NCAR / DART