Summary of Progress

[AndyHoggANU]

I am starting this issue to highlights the choices and compromises we had to make to get splitvar to operate across 3 different resolutions for different length simulations. The differences between each case are highlighted below.

First, a few general comments:

• The biggest obstacle we encountered was tuning the memory requirements. This often required splitting up the script into different packets, or running different-sized interactive/submitted qsub jobs.
• The second most common error was when variables (or dimensions) changed during the run. This created memory errors which were ...errr... dificult to disentangle from memory errors.
• Missing files also created headaches.
• Need to decide when to use --atefrombounds to get filenames right.
• One needs be careful that you are loading the right version of the splitvar package!!

1 degree

The script for this can be found in the mom_10 directory:

• Generally, I ran this on an interactive submission on raijin, with several cores and 192Gb
• We use parallel in bash to do these runs
• The first 4 cycles of the simulation are in one directory, and the last cycle in a different directory - these had to be processed independently.
• Each ice cycle was processed independently, just because the list of filenames became too long to process in a single command!

0.25 degree

Look in mom_025:

• This could also be done in a large memory interactive submission
• 2D variables sailed through, but they needed to be split up into 5 cycles
• 3D variables took longer, but can be done.
• ice suffered from a corrupted file in the 3rd cycle - worked when this file was removed.

0.1 degree

Look in mom_01:

• This is set up to run interactively and spawn off a single high-memory job for each variable.
• Tuning the run time and the memory took some effort!
• Variables are patchily saved, and splitvar doesn't cope well with this, even if you are not using that variable.
• Ice needed to exclude grid variables and reload via the -a flag to get through.