address reviewer comments

[orbeckst]

We got two very constructive reviews at scipy-conference/scipy_proceedings#290

I am creating a todo list from the reviewers' comments.

@mkhoshle and @iparask please have a look. If there are points in the list with your name, please have a look and provide some answers.

When editing the paper, please do one commit per question addressed and check the appropriate box.

DEADLINE for resubmission (i.e., addressing all comments): July 7

General

• This paper shows the experience of parallelizing a standard parallel programming computation with Dask. It goes through the experience of identifying and optimizing a variety of issues that arise in parallel and distributed computing in common practice. These issues (network I/O, stragglers, communication/scheduler overhead) are important and rarely discussed, so it is nice to see them show up in a published work.
• [ ] This Paper appears to be above the 8 page limit maybe we can keep the size...? #8
• This paper does a good job of being reproducible. It includes code, data, and analysis

Nit-picky feedback

• [ ] I have not seen NetCDF abbreviated to NCDF before. I recommend adding back in the "et" in "net" to stick to convention. Change @mkhoshle
• [ ] Figures 3/4 #9
  • [ ] Text is somewhat small. Increase font size
  • [ ] I'm uncertain what the axes mean. I interpret "IO Per frame" as amount of time spend to read all blocks, is this correct? Is the "number of blocks" also the same as the number of processes used? If so then why are these tied together? If my understanding is correct then I would expect speedups to visually manifest as a downward slope. If convenient it might be useful to switch axes to show bandwidth (MB/s) on the Y-axis instead. Clarify/discuss @mkhoshle
  • [ ] If number of blocks is the same as number of cpus then I recommend changing axis labels to cpus instead of blocks. I believe that this will be more readily understood by readers. Clarify, possibly change everywhere
• [ ] When using local HDD/SSD in a distributed setting how do you assign files to workers? describe @mkhoshle
• [ ] If all DCD is doing is ripping bytes from disk without any processing then I wouldn't expect much dependence on parallelism at all. It should be entirely determined by the I/O bandwidth to the CPUs. discuss #10
• [ ] The y-axes in figure 5 should probably be the same. As is one of them has amaximum of 10e4 while the others have maxima of 10e3. This is somewhat misleading at first glance. Make axes same @mkhoshle
• [ ] Many Figures for 300x/600x seem to have identical captions. it might be wise instead to include these pairs of figures into a single large figure with a 2x3 grid of images and a single caption. This would also allow you to more easily highlight differences between the 300x and 600x runs. Make panel #9
• [ ] The similarity of times measured by your infrastructure and the dask web interface does not seem like an important point to focus on. I might skip it and exclude it from table 1. Need to discuss between us, possibly remove as suggested @mkhoshle @orbeckst #11
• [ ] Figures labels might be better if they avoided internal variable names. For example Total time t_{total} (s) might be replacec with Total time (s) ?... check, possibly change #12
• [ ] Much of this study seems to be about strong scaling, seeing how fast we can run the same computation on more resources. As data sizes increase presuambly weak scaling is also useful. Seeing how much independent data we can churn through on larger systems. Are MDAnalysis workloads amenable to this? Or do you process through one frame before another which stops embarrassingly parallel work across time? discuss briefly (strong scaling is our main objective at the moment) #13

Examining Scheduler Throughput

• [ ] Fig 15: y-axis should be tasks per second? please check, @iparask
• [ ] I am somewhat surprised to see the distributed scheduler manage 40,000 tasks per second. please check, @iparask
• [ ] I recommend not capitalizing multiprocessing, multithreading, scheduler, cluster, or distributed. change
• [ ] It might be useful to have a link to your dask scheduler plugin from your paper. add GitHub link for plugin – @iparask ?

Remove jargon

• [ ] I missed having a reference explaining a concept of "optimal structural superposition of atoms". explain, check reference @orbeckst

[iparask]

The y-axis of figure 15 is tasks because it shows how many tasks on average finished in every second of the execution. I will change it to tasks per second if that is preferable.

[mkhoshle]

Yes, please change it to task per second!

[orbeckst]

@mkhoshle and @iparask I think I am done. Please have a look for final checks.

A PDF is attached here: paper.pdf

I did quite a bit of rewriting and reorganizing to improve the logic and the flow and to shorten it a bit. Please check that everything is still true...

We also have to write a short "rebuttal" with point-by-point comments #15.