NIST Review

[ramess101]

@ejmaginn @drroe @dcarls0n @jrelliottoh

I have attached the first NIST reviewer's remarks. All of the changes are minor, but I think two of them are pretty important. First, adding multiple time origins to the checklist and including a bit more discussion about this point. Second, more details about bootstrapping. Specifically, how large the set should be relative to the number of replicates.

EMD_NIST_reviewer1_comments.pdf

[drroe]

I can't seem to download the PDF - the link appears to just refer to this page. Anyone else seeing that?

[ramess101]

@drroe

Sorry, I did not wait for the download to finish. It should work now.

[ejmaginn]

Rich,

I can’t seem to download this - help?

Ed

Edward Maginn Dorini Family Professor and Department Chair Department of Chemical and Biomolecular Engineering University of Notre Dame Notre Dame, IN 46556 USA cbe.nd.edu ed@nd.edu @ejmaginn view my papers: https://goo.gl/kgMMBe

On May 9, 2018, at 1:47 PM, Richard Messerly notifications@github.com wrote:

@ejmaginn https://github.com/ejmaginn @drroe https://github.com/drroe @dcarls0n https://github.com/dcarls0n @jrelliottoh https://github.com/jrelliottoh I have attached the first NIST reviewer's remarks. All of the changes are minor, but I think two of them are pretty important. First, adding multiple time origins to the checklist and including a bit more discussion about this point. Second, more details about bootstrapping. Specifically, how large the set should be relative to the number of replicates.

Uploading EMD_NIST_reviewer1_comments.pdf… applewebdata://547F4C20-F0C5-40FB-866F-56553BCCB0F3 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ejmaginn/TransportCheckList/issues/9, or mute the thread https://github.com/notifications/unsubscribe-auth/AJKiXfrKoX0sUUzDmgleQzvB7c-w9AmBks5twyuXgaJpZM4T4ulD.

[ramess101]

@ejmaginn

Here it is again

EMD_diffusivity_viscosity_restructured.pdf

[ejmaginn]

Got it - thanks. Ed

On May 9, 2018, at 10:13 PM, Richard Messerly notifications@github.com wrote:

@ejmaginn https://github.com/ejmaginn Here it is again

EMD_diffusivity_viscosity_restructured.pdf https://github.com/ejmaginn/TransportCheckList/files/1990064/EMD_diffusivity_viscosity_restructured.pdf — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ejmaginn/TransportCheckList/issues/9#issuecomment-387930642, or mute the thread https://github.com/notifications/unsubscribe-auth/AJKiXduRodGM0AIfZAqAKk2gOnnh6lpZks5tw6I2gaJpZM4T4ulD.

[ramess101]

@drroe @ejmaginn @jrelliottoh @dcarls0n

We are still waiting to hear back from the second NIST reviewer. If you have not already done so, please look over the previous reviewers comments. If you object to any of them, make it known soon. I plan on implementing the first reviewers changes tonight.

[drroe]

The reviewer’s comments (from what I remember, don’t have them in front of me ATM) were all minor and reasonable, so I don’t have anything to add. Let me know if I can assist in clarifying anything.

-Dan

On Tue, May 15, 2018 at 3:39 PM Richard Messerly notifications@github.com wrote:

@drroe https://github.com/drroe @ejmaginn https://github.com/ejmaginn @jrelliottoh https://github.com/jrelliottoh @dcarls0n https://github.com/dcarls0n

We are still waiting to hear back from the second NIST reviewer. If you have not already done so, please look over the previous reviewers comments. If you object to any of them, make it known soon. I plan on implementing the first reviewers changes tonight.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ejmaginn/TransportCheckList/issues/9#issuecomment-389288329, or mute the thread https://github.com/notifications/unsubscribe-auth/AJrJtyqecZqj9tEnAjX8jIrGUOEV7qBbks5tyy7dgaJpZM4T4ulD .

[ramess101]

@drroe @ejmaginn @jrelliottoh @dcarls0n

I have implemented the first reviewer's comments. The second reviewer just got back to me. Here are his remarks:

It is worthwhile to promote care for uncertainties in computational molecular science as it is routine for experimenters. The Guide to the Expression of Uncertainty in Measurement and related documents as well as the NIST Uncertainty Machine may be helpful to be considered and quoted in this manuscript.

References [1] and [2] should include URLs to LAMMPS and GROMACS.

The book of Haile should be quoted much more prominently, i. e. earlier in the text, because it is the most pedagogical tutorial of the molecular dynamics method and the best explanation of autocorrelation functions.

The URLs of references [12], [13], and [14] should have tildes ~ instead of the spaces they contain to be functional.

Reference [15] is just a two-page note. Is it that significant?

Line 4 and 5 in the caption of Figure 3: should "length, i. e. trajectory time" not be plural?

Line 4 in the caption of Figure 6: should "number" not be plural?

Page 14, last line of right column, correct "heurestic" to "heuristic".

Page 17, line 3 of last paragraph in left column: change "as" to "are".

References: In some citations the DOI and the URL of the DOI are given. Giving only the URL is sufficient and they should be unified to http://doi.org…

[ramess101]

Here are my responses to each comment.

It is worthwhile to promote care for uncertainties in computational molecular science as it is routine for experimenters. The Guide to the Expression of Uncertainty in Measurement and related documents as well as the NIST Uncertainty Machine may be helpful to be considered and quoted in this manuscript.

I think we could reference the GUM as well as the LiveCoMS best practices for reporting uncertainty manuscript.

References [1] and [2] should include URLs to LAMMPS and GROMACS.

OK,

The book of Haile should be quoted much more prominently, i. e. earlier in the text, because it is the most pedagogical tutorial of the molecular dynamics method and the best explanation of autocorrelation functions.

This is an important text, but the explanation about transport properties is not very developed or straight forward, in my opinion. And the focus for viscosity is primarily on NEMD. However, I will include it in our list of suggested references to read, specifically, pages 189-192, 485-488,

The URLs of references [12], [13], and [14] should have tildes ~ instead of the spaces they contain to be functional.

I already fixed this.

Reference [15] is just a two-page note. Is it that significant?

This is Ref [17] now. And yes, I do believe this is a very significant note, albeit brief.

Line 4 and 5 in the caption of Figure 3: should "length, i. e. trajectory time" not be plural?

Seems reasonable.

Line 4 in the caption of Figure 6: should "number" not be plural?

Sure.

Page 14, last line of right column, correct "heurestic" to "heuristic".

Already corrected this.

Page 17, line 3 of last paragraph in left column: change "as" to "are".

Already corrected this.

References: In some citations the DOI and the URL of the DOI are given. Giving only the URL is sufficient and they should be unified to http://doi.org…

I will fix this.

[ramess101]

@drroe @ejmaginn @jrelliottoh @dcarls0n

Let me know if you have any further suggestions in response to these comments. They are all pretty minor. So I will implement them today and submit for the final NIST review by tomorrow. It should be approved by Monday.

[ejmaginn]

This all seems reasonable to me.

Ed

On May 16, 2018, at 10:23 AM, Richard Messerly notifications@github.com wrote:

@drroe https://github.com/drroe @ejmaginn https://github.com/ejmaginn @jrelliottoh https://github.com/jrelliottoh @dcarls0n https://github.com/dcarls0n I have implemented the first reviewer's comments. The second reviewer just got back to me. Here are his remarks:

It is worthwhile to promote care for uncertainties in computational molecular science as it is routine for experimenters. The Guide to the Expression of Uncertainty in Measurement and related documents as well as the NIST Uncertainty Machine may be helpful to be considered and quoted in this manuscript.

References [1] and [2] should include URLs to LAMMPS and GROMACS.

The book of Haile should be quoted much more prominently, i. e. earlier in the text, because it is the most pedagogical tutorial of the molecular dynamics method and the best explanation of autocorrelation functions.

The URLs of references [12], [13], and [14] should have tildes ~ instead of the spaces they contain to be functional.

Reference [15] is just a two-page note. Is it that significant?

Line 4 and 5 in the caption of Figure 3: should "length, i. e. trajectory time" not be plural?

Line 4 in the caption of Figure 6: should "number" not be plural?

Page 14, last line of right column, correct "heurestic" to "heuristic".

Page 17, line 3 of last paragraph in left column: change "as" to "are".

References: In some citations the DOI and the URL of the DOI are given. Giving only the URL is sufficient and they should be unified to http://doi.org… http://doi.org%E2%80%A6 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ejmaginn/TransportCheckList/issues/9#issuecomment-389537103, or mute the thread https://github.com/notifications/unsubscribe-auth/AJKiXeUMBqbK8AeHDoO3Un1twhSRRHm5ks5tzDZOgaJpZM4T4ulD.

[ejmaginn]

I think you have responded fine here. I agree with your comment on Haile’s book. Ed

On May 16, 2018, at 10:46 AM, Richard Messerly notifications@github.com wrote:

Here are my responses to each comment.

It is worthwhile to promote care for uncertainties in computational molecular science as it is routine for experimenters. The Guide to the Expression of Uncertainty in Measurement and related documents as well as the NIST Uncertainty Machine may be helpful to be considered and quoted in this manuscript.

I think we could reference the GUM as well as the LiveCoMS best practices for reporting uncertainty manuscript.

References [1] and [2] should include URLs to LAMMPS and GROMACS.

OK,

The book of Haile should be quoted much more prominently, i. e. earlier in the text, because it is the most pedagogical tutorial of the molecular dynamics method and the best explanation of autocorrelation functions.

This is an important text, but the explanation about transport properties is not very developed or straight forward, in my opinion. And the focus for viscosity is primarily on NEMD. However, I will include it in our list of suggested references to read, specifically, pages 189-192, 485-488,

The URLs of references [12], [13], and [14] should have tildes ~ instead of the spaces they contain to be functional.

I tried to fix this, but the best I could do was get the tildes ~ to appear without an active link. I read that I can use %7E instead, but that made the URLs to long to fit on one line.

Reference [15] is just a two-page note. Is it that significant?

This is Ref [17] now. And yes, I do believe this is a very significant note, albeit brief.

Line 4 and 5 in the caption of Figure 3: should "length, i. e. trajectory time" not be plural?

Seems reasonable.

Line 4 in the caption of Figure 6: should "number" not be plural?

Sure.

Page 14, last line of right column, correct "heurestic" to "heuristic".

Already corrected this.

Page 17, line 3 of last paragraph in left column: change "as" to "are".

Already corrected this.

References: In some citations the DOI and the URL of the DOI are given. Giving only the URL is sufficient and they should be unified to http://doi.org… http://doi.org%E2%80%A6 I will fix this.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ejmaginn/TransportCheckList/issues/9#issuecomment-389545732, or mute the thread https://github.com/notifications/unsubscribe-auth/AJKiXalh4mrsQK5qkR2rgWddMcz5pBO2ks5tzDvXgaJpZM4T4ulD.

[ramess101]

@drroe @ejmaginn @jrelliottoh @dcarls0n

I just wanted to let everyone know that there was a paperwork mishap at NIST. So the manuscript was not reviewed this past Monday, but it will be reviewed on 5/29 (a Tuesday because there is a federal holiday on Monday).

Also, I have included some of the syntax changes that were recommended by Ken Kroenlein of TRC. Most likely there will be a few more non-consequential recommendations in the final two editorial reviews. I will let you know if there is anything important enough for us all to discuss.

I expect to submit the manuscript to LiveCoMS on May 30th.

[ejmaginn]

@ramess101 thank you - sounds like a good plan.

[ramess101]

@drroe @ejmaginn @jrelliottoh @dcarls0n

Our manuscript has passed the final NIST review. There are a few comments that I wanted to address below.

[ramess101]

On page 2 of the manuscript, below Equation 1, we state that "\xi is the perturbation in the Hamiltonian associated with the particular transport property under consideration." However, I believe that statement was orginally written for NEMD methods. EMD does not include any "perturbation in the Hamiltonian." Please correct me if I am misinterpretting this statement. Unless someone else has a better explanation, I am going to replace this statement with "\xi is the mechanical variable associated with the particular transport property under consideration." Does that sound good to everyone?

[ramess101]

On page 3, section 4.1.2, paragraph 2, we state that "the uncertainty is inversely proportional to the square root of the number of replicates." Is this uncertainty supposed to be the variance, standard deviation, standard error of the mean, or expanded confidence interval?

[ramess101]

@drroe @ejmaginn @jrelliottoh @dcarls0n

Is this uncertainty supposed to be the variance, standard deviation, standard error of the mean, or expanded confidence interval?

Here is the reason for my confusion.

On page 526 of Frenkel and Smit we read:

It simply states the well-known fact the variance in a measured quantity is inversely proportional to the number of uncorrelated measurements.

However, I believe it is really the standard error of the mean or the 95% confidence interval that decreases as Nreps^(-1/2). Although the variance is calculated from

Where it appears to be inversely proportional to N, the summation in the numerator increases proportional to N as well. The same argument explains why the standard deviation does not change with N.

By contrast, the standard error of the mean is defined as:

Which is inversely proportional to the square root of N. I believe this is what our original statement was meant to convey. Furthermore, we can conclude that the variance of the mean is inversely proportional to N, which is likely what Frenkel and Smit are trying to say.

To help demonstrate all of this, if you generate data from a normal distribution and repeatedly compute the standard deviation, variance, standard error of the mean, and 95% confidence interval, you see that the estimate for the standard deviation and the variance is constant with respect to Nreps while the standard error of the mean and 95% confidence interval decrease as Nreps^(-1/2):

Unless anyone disagrees, I am going to change this statement to, "The uncertainty (specifically, the standard error of the mean) is inversely proportional to..."

[ejmaginn]

I think you are right Rich - this is the standard error.