Paper: Predictive modelling of toxicity resulting from radiotherapy treatments of head and neck cancer (J. Dean)

[NelleV]

Thank you for the submission! Review process will start August, 20th.

[bertiniteam]

Jamie, I will be one of the reviewers for your paper. I will post comments here no later than September 20. Best, Dan Bates (Colorado State University)

[katyhuff]

I'll also review this paper. (same deadline, September 20.)

[bertiniteam]

REVIEW

This article describes the use of Python (particular NumPy and SciPy) for the manipulation of radiotherapy data and computation in the context of the treatment of head and neck cancers. More specifically, the authors used various Python modules to generate 3D maps of the dose distribution delivered to the mucous lining of the throat (pharyngeal mucosa), combined these maps with dosing data (radiation treatment plans), and used machine learning techniques to determine which sorts of treatment ultimately led to decreased swallowing function, particularly severe dysphagia, which requires the use of a feeding tube. There are several useful images throughout the article, adding to the quality of the exposition.

General impression

This paper is very well written, and I found the application to be interesting and well-presented. To my novice Python eyes, the authors seem to include plenty of details about how they used Python in their work. While the clustering approach they chose (support vector machines with radial basis functions) ultimately failed to provide a reasonable amount of predictive power, they have constructed what seems to be a good computational platform for further research in this direction. It is good, too, that they seem to have a handle on the fundamental problem with their approach - spatial information is necessarily discarded during data manipulation, but spatial information is likely very important for classification.

I have no major concerns about the inclusion of this paper in these proceedings, aside from a few minor concerns, below. Thank you, authors, for the enjoyable and professionally prepared paper. I hope that your next attempt at classification goes well.

Minor comments

1. Introduction, line 2: "(swallowing function)" - should this be swallowing dysfunction?
2. Introduction (the second one): Drop "to be explored" at the end.
3. Preparation of RT Plan Data: I have no experience in your area, but is there any risk of error in the automatic renaming of structures named by radiation oncologists? In particular, is there any potential for ambiguity? I might be good to include a few words about this.
4. Generating 3D Dose Maps: I understand the difficulty of having coarser dose grids than image grids, but I wonder whether linear interpolation is a reasonable way to increase the resolution. Does this accurately model the administration of the doses, or does the radiation actually follow the contour of the PM in some other way?
5. Statistical Modelling: Was the "(...)" after "alcohol status" intentional?
6. I appreciate your honesty in the Discussion section. I imagine the results were disappointing, but this work still represents significant effort that could eventually (with different computational methods) be very valuable. I wonder how well your code will generalize to other treatments and other maladies in other parts of the body. I would be interested to read your thoughts on this in the discussion section, though this is not required.

Nice work!

[katyhuff]

Review

This paper describes the software analysis pipeline behind an effort to characterize and predictively model the relationship between medical radiation dose patters and dysphagia severity. With an intent to improve radiation planning techniques, this work employs modern algorithms to real-world data. The use of Python to clean the data, conduct statistical analysis, employ machine learning techniques, and plot three dimensional structures was well described. Code snippets were informative and demonstrative of good practice. The machine learning techniques applied to the dataset were described in sufficient detail, and the results were clearly explained. Furthermore, the authors were candid concerning the weaknesses of their predictive model, which is to be commended. Finally, promising future efforts were outlined.

Recommendation

I recommend this paper be accepted with very minor corrections.

Overall Impression

This is a very well-written article with strong relevance to the topic of the conference, namely, the scientific use of python. I am impressed with the clarity of explanation and the overall structure of the paper as well as its goals and execution. However, I would strongly prefer that the discussed code be transparently available on line. In the event that this transparency is impossible, I would prefer that the authors address the method by which the code may be reviewed by experts. In a similar vein, I would be interested in an additional section on the quality assurance measures undertaken with this software development project. In particular, the existence or absence of unit tests could strongly influence my trust in the algorithm implementations. Again, however, I think overall technical quality of the article is excellent and I recommend its publication with or without adherence to the opinions above. Requested minor corrections are in the following section.

On a personal note, this work seems to address an issue of exceptional importance to radiotherapy in medicine. It is inspiring to read about this work!

Suggested Minor Corrections

• I am very impressed at the error free nature of the text. However, I do have a number of suggested minor hyphen and comma insertions to suggest. These are not mandatory corrections - only suggestions. This marked-up copy of your text contains those suggestions here. If you would prefer those suggestions as line-comments in the pull request, please let me know.
• In the introduction, I also caught the swallowing function vs. swallowing dysfunction typo. As noted by the other reviewer, it should be fixed to 'swallowing dysfunction' before publication.
• In general, the first instance of any acronym should be expanded. In publications within a closely-knit research field, this is not as crucial as it is in an audience from the disperse scientific backgrounds that construct the SciPy audience. Please consider expanding the first instance of any acronyms.
• During my reading, I did not find a citation for your data from the Royal Marsden NHS Foundation Trust. I recommend explicitly citing the source of your data. If a digital object identifier (DOI) is available, that would be ideal.
• The first sentence in the statistical modeling section should be rephrased. It is currently ambiguous, I believe. It says you extracted those "volumes equal to or less than 10Gy to 80Gy". Do you mean "V<=80" or "10<=V<=80" ? The way it's currently phrased seems to say both, which can't be true.
• The citation you use for numpy is a good paper, but I wonder if citing the main developers and the numpy source code would be more appropriate. Perhaps something like : "Travis Oliphant, Charles Harris, Jarrod Millman, Ralf Gommers, Robert Kern, Numerical Python (NumPy) v1.X.x. http://sourceforge.net/projects/numpy/ or numpy.scipy.org"

Additional Comments

Thoughts that this paper raised, which may be of interest to the author:

• I am interested in whether you noticed any distortions caused by the linear interpolation step in generating the 3D dose maps. Perhaps more sophisticated interpolation could improve the behavior of your method?
• You note that the difference between planned and actual patient irradiation may be thwarting your method. Do you have any notion of the likely magnitude of that effect? It may be worth estimating the importance fairly robustly before pursuing improvements along that avenue. I suspect, and you seem to also, that the problems lie in the covariant parameters.

[jamiedean]

We wish to thank you both for your insightful comments. We believe that addressing your remarks will strengthen the manuscript and will endeavor to revise the article promptly. Thanks again for your thorough reviews.

Kind regards,

Jamie Dean (on behalf of all of the authors)

[pdebuyl]

Dear @danbates and @katyhuff , thank you for your reviews.

Dear @jamiedean , please propose modifications addressing the comments made in the reviews.

Your updates should arrive for mid-october so that the reviewers can assess the update before the end of october.

[jamiedean]

Dear Dr Dan Bates and Dr Katy Huff,

We revised the manuscript based on your helpful comments and resubmitted it last week. We have addressed all of your comments apart from the one regarding the public release of code. As this is part of an ongoing project we will not be releasing the code at the present time (this is inline with current common practice in the radiation oncology community). However, it is something we will strongly consider at the conclusion of the project. We hope that you agree that the manuscript is now stronger as a result of the changes. Please let me know if you would like us to make any further alterations.

Kind regards,

Jamie Dean (on behalf of all of the authors)

[katyhuff]

Great work!

[bertiniteam]

I agree -- this looks very good. Nice job!

[pdebuyl]

Thanks all for the paper and reviews! The paper will be included after routine checks on the layout, etc.

[pdebuyl]

Dear @jamiedean

The abstract of the paper is too long and structured with intermediate headings. It is necessary to reformat it so that it is composed of regular paragraphs (without leading paragraph title) only. Having an introduction and a conclusion in the abstract is somewhat excessive.

Regards, Pierre

[jamiedean]

Dear @pdebuyl

We have shortened the abstract and removed the headings. We hope this is now acceptable. Please let us know if you'd like us to make any further changes.

Kind regards,

Jamie

[pdebuyl]

Thanks for the quick update, the updated abstract fits very well.

No further change is needed. I suggest, if reasonable, to regenerate figures 3, 4, 5, 6 and 7 as pdf to benefit from a better printing. As your png are of acceptable quality I will merge your paper already and you are free to submit a new pull request against the main repository if/when pdf figures are available. That is until the full proceedings are uploaded indeed.

Regards,

Pierre