search

dmzuckerman / Sampling-Uncertainty

Best Practices article intended for LiveCoMS
36 stars 5 forks source link

Catch all / Remaining Reviewer Comments #48

Closed dwsideriusNIST closed 6 years ago

dwsideriusNIST commented 6 years ago

Reviewer comments:

"Some pitfalls could be mentioned. Students often take the average from a trajectory and then the standard deviation and assume this is the "error bar"! They don't realize that the standard deviation here is just a measure of the fluctuations, which is just related to system size. It might be worth mentioning somewhere."

"The density of information contained in different sections can vary strongly. That makes the manuscript partially harder to read. While sections 1-5 are rather stretched out and new information is presented slowly, section 6 contains much more information in much less text. Moreover, while in sections 1-5 also rather basic concepts are introduced with care, in section 6 more advanced concepts are just introduced very quickly or not even explained at all. Examples for this are Voroni cells mentioned at the beginning of section 6.1 on page 12 or PCA mentioned on page 13. This makes section 6 partially hard to understand. Figure 6 helps to understand this section, but is not referenced in the text."

"p2: Maybe one should also include software/programming errors (see e.g. Ref 1, and a number of other recent references reporting "bugs" in heavily-used MD codes like GROMACS)"

agrossfield commented 6 years ago

In response to the first issue, I added a sentence to the definition of standard deviation, reminding the reader that std dev is a measure of the width of the distribution and not a measure of statistical uncertainty.

agrossfield commented 6 years ago

In response to the third issue, I added a sentence to the scope section, saying we don't address bugs and implementation errors.

I didn't add a reference to software best practices -- isn't there an article on that coming from the same workshop that led to this article?

dwsideriusNIST commented 6 years ago

I updated the footnote on pg 2 to reference the best practices papers regarding bugs and physical validation.

dwsideriusNIST commented 6 years ago

I'm inclined to ignore the issue of information density; we wrote each section to be appropriate for its content, which does lead to asymmetries, but there is a purpose in it.

agrossfield commented 6 years ago

In that case, I think we can close this issue. Objections?