HPC RSE: improving the parallel scaling of Python

[Edward-RSE]

This PR merges in changes related to the second HPC RSE project at the University of Southampton. There were X aims to this project:

1. Improve the parallel scaling of Python by parallelising serial bottleneck in the ionisation calculation (e.g. wind_rad_init(), wind_cooling() and update_wind())
2. Re-factor, re-structure and optimise MPI communication
3. Improve memory access patterns for memory intensive parts of the program
4. Implement additional unit testing where appropriate

This is a large pull request and it has touched a lot of files. There have been a lot of changes to the source code and I have also re-normalized the style using run_indent.py which touched files I did not work on. The directory structure of unit test data has also been modified, which has resulted in file changes.

Detailed Changes

• Refactored and centralised MPI communication
  • Four new files: communicate_macro.c, communicate_plasma.c, communicate_spectra.c and communicate_wind.c
  • The size of packed buffers are calculated using the library function MPI_Pack_size
  • The maximum number of cells required for a buffer is calculated using a new function get_max_cells_per_rank
• Parallelised wind_cooling()
• Refactored and parallelised wind_rad_init()
  • Split into two smaller functions init_plasma_rad_properties() and init_macro_rad_properties()
  • The function init_macro_rad_properties() has been parallelised, but only when computing the macro atom recombination coefficients
• Refactored and parallelised define_wind()
  • Most of the grid creation is done in parallel. What remains in serial is defining the coordinate grid and sanity checks
  • Split into numerous smaller functions, which constructs each “grid” separately
  • Changed the signature of many related functions to take in a single cell, rather than the entire grid
• Created a new function setup_atomic_data()
  • Reading in atomic data is no longer deeply embedded into the ionisation initialisation routines. This is done to make unit testing possible when a model needs to be read in.
  • Reading in the atomic data is now done slightly later in the program
• Added new unit tests to test define_wind()
  • New test data and tests added to check spherical and cylindrical models. These unit tests are very high level and test define_wind() in its entirety. Ideally, more tests should be added which test smaller units of the grid creation process.
  • New helper functions which can read a parameter file and set up the basics for Python to create a grid to be tested on
• Cleaned up the Makefile in the source directory
  • Removed unnecessary duplication of source files for each binary. The source files of Python are defined in a single variable, which is re-used in the other binary files. Pattern substitution is used to define the object files, instead of re-writing the source files but with .o extensions.
  • New debug flags are added for GCC and Linux systems related to Valgrind
  • C99 standard is enforced by the compiler and -Wno-deprecated-non-prototype has been added to stop complaints raised by recent Clang and GCC compilers
  • Added a new make version recipe to quickly make version.h
• Modified the GitHub actions
  • Put the unit test framework in a more sensible place
  • Removed redundant re-compilations of the program, which means the action is a little quicker
• New regression test added to test imported models

[Edward-RSE]

FYI, the checks fail on the Balmer test which is also happening on the dev branch.

[kslong]

Obviously, we want to wait to merge this until I return from Tahiti in about 10 days from now, given that this merge will refactor a good bit of the code.

[jhmatthews]

@Edward-RSE the dev branch is passing tests since Knox reverted the integration changes, so is that weird? Or does it mean a pull/rebase is needed?

[Edward-RSE]

Oops, it's actually failing in fixed temperature mode for this model. Brb, fixing and adding a new test

[Edward-RSE]

In the end the test was failing because of the Balmer test using a shell wind, not because of fixed temperature mode (which seems to work, although it would be good to get a proper test model for that). However, now I'm having an issue with some memory management at the end of the program so I'm going to convert this PR back to draft until that's fixed.

I also found another issue to do with allocating memory for domains in the utility programs, which wasn't caught in the test suites. It would be good to also test the entire menagerie of programs.

[Edward-RSE]

This should be good to go now. There was an issue relating to setting up 1D models, which is now fixed.