MPI parallelisation

[aosprey]

Plan for parallelising the code with MPI:

Control code:

• Create a module to hold parallelisation routines: mckpp_mpi_control
• Sort out where MPI and XIOS initalization and finalization is done
• Create a log file per MPI rank (we could also have an option to just have root process write logs)
• Make sure mckpp_abort stops all ranks
• Report timer statistics across ranks

Domain decomposition:

• 1D domain decomposition over npts, define array size npts_local.
• Identify whether some fields need to be held globally.

Parallel routines:

• Broadcast field
• Scatter fields - 1d, 2d, 3d, etc arrays
• I don't think we need a gather

XIOS:

• Define decomposition

Parallel netcdf reads:

• Have a root rank read data, and collapse x,y dimensions into npts.
• Scatter fields along npts.

[aosprey]

What makes the parallelisation tricky is that we want to parallelise over npts but this dimension is contiguous in memory, since the arrays are dimensioned array(npts,nz). Therefore scattering any 2d (or 3d fields) requires extracting non-contiguous chunks of data and packing into a send buffer. If the dimensions were reveresed and nz was contiguous, the scatter could be done with an MPI_scatter routine directly (I think).

It would be better to fix this first #15

[aosprey]

Some progress on parallelisation has been made in the mpi branch.

[aosprey]

We may want to have a data structure to hold global fields (as in CAM routines), e.g. kpp_global_fields.