Review: 3. Writing Parallel Algorithms in MPI

[Edward-RSE]

This issue is to track comments and changes for this episode.

[Edward-RSE]

See https://github.com/Southampton-RSG-Training/dirac-intro-to-mpi/issues/8#issuecomment-1607508510 and https://github.com/Southampton-RSG-Training/dirac-intro-to-mpi/issues/9#issuecomment-1607610932 on how I think this episode and the first two episodes could be re-shuffled and re-consolidated into two episodes.

[Edward-RSE]

Sorry, working from the bottom up for the episode, so to address the last section first...

Using MPI to Split Work Across Processes

The section is titled "splitting work across processes" but we don't show that or explain how to do it in the text (or in the next episode).

I think we really need to explicitly show how we'd split some (trivial) work up using the rank number and rank size, e.g. something like this should be shown and explained,

#define NUM_ITERATIONS 1000

int my_rank;
int num_ranks;

MPI_Comm_rank(MPI_COMM_WORLD, &my_rank);
MPI_Comm_size(MPI_COMM_WORLD, &num_ranks);

// calculate how many iterations each rank will deal with
int iterations_per_rank = NUM_ITERATIONS / num_ranks;
int rank_start = my_rank * iterations_per_rank;
int rank_end = (my_rank + 1) * iterations_per_rank;

// catch cases where the work can't be split evenly 
if (rank_end > NUM_ITERATIONS) {
    rank_end = NUM_ITERATIONS;
}

// each rank is dealing with a subset of the problem 
for (int i = rank_start; i < rank_end; ++i) {
    // code to work on that subset of data
}

We also have another code example from DiRAC (https://github.com/DiRAC-HPC/HPC-Skills-Pi/blob/main/Scaling/MPI/mpi_pi.c) which splits work a bit differently (there isn't a license for this example, but I'm sure Richard won't mind us re-using it).

[Edward-RSE]

Continuing with the review from the top now, just to be confusing.

Serial and Parallel Execution

• We have two analogies in the lesson now. Painting a room and making a car. I think sticking with one real life analogy would make for a better narrative. Thoughts? Is it worth the work?

Data dependency

• Should we change the pseudocode into C?

• I think the solution text in the "Serial and Parallel Regions" exercise could do with a bit more text. Something like: "In the first loop, every iteration depends on data from the previous two so we cannot parallelise this because of the data dependency."

Parallel Paradigms

• We're using the term "rank" before defining what it means. It isn't first mentioned until nearer the end of the lesson. I suggest we use "process" instead of rank, which is something we've already talked about.

Algorithm Design

• Same comment about using "rank".

Domain Decomposition

• Let's also mention something about gridded codes (fluid dynamics, for example)

[steve-crouch]

Completed.