[Feature Request] fix ave/atom etc should produce standard deviation

[andeplane]

When using the ave/atom (and similar averaging commands) it makes sense to be able to produce the standard deviation for the quantity.

For instance stddev yes will add N extra columns with the standard deviations for each N values.

What is your view on this?

[rjones-sandia]

I think variances would be super useful given the stat mech context. I wrote a version of fix ave/time to use an online/real time estimate of the variance. I'd would like to add that to lammps proper

Reese Jones rjones@sandia.govmailto:rjones@sandia.gov 925 294-4744 "What's gone and what's past help should be past grief" - Paulina, The Winter's Tale.

On Nov 3, 2016, at 12:58 PM, Anders Hafreager notifications@github.com<mailto:notifications@github.com> wrote:

When using the ave/atom (and similar averaging commands) it makes sense to be able to produce the standard deviation for the quantity.

For instance stddev yes will add N extra columns with the standard deviations for each N value.

What is your view on this?

You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/lammps/lammps/issues/251, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AHZBhS-alsrTLxnMxN1i4V7lXek-Jtyeks5q6jyxgaJpZM4Ko2Ho.

[athomps]

I agree. Providing this as an option to fix ave/time would be very beneficial. I never again want to figure out the correct form of this expression:

varx = (sumxsq - sumx*sumx/n)/n

Aidan

  Aidan P. Thompson
  01444 Multiscale Science
  Sandia National Laboratories
  PO Box 5800, MS 1322      Phone: 505-844-9702
  Albuquerque, NM 87185     Fax  : 505-845-7442
  E-mail:athomps@sandia.gov Cell : 505-218-1011

From: rjones-sandia notifications@github.com Reply-To: lammps/lammps reply@reply.github.com Date: Thursday, November 3, 2016 at 2:02 PM To: lammps/lammps lammps@noreply.github.com Subject: [EXTERNAL] Re: [lammps/lammps] fix ave/atom etc should produce standard deviation (#251)

I think variances would be super useful given the stat mech context. I wrote a version of fix ave/time to use an online/real time estimate of the variance. I'd would like to add that to lammps proper

Reese Jones rjones@sandia.govmailto:rjones@sandia.gov 925 294-4744 "What's gone and what's past help should be past grief" - Paulina, The Winter's Tale.

On Nov 3, 2016, at 12:58 PM, Anders Hafreager notifications@github.com<mailto:notifications@github.com> wrote:

When using the ave/atom (and similar averaging commands) it makes sense to be able to produce the standard deviation for the quantity.

For instance stddev yes will add N extra columns with the standard deviations for each N value.

What is your view on this?

You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/lammps/lammps/issues/251, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AHZBhS-alsrTLxnMxN1i4V7lXek-Jtyeks5q6jyxgaJpZM4Ko2Ho.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/lammps/lammps/issues/251#issuecomment-258257531, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AKVnIHkqUN6lda7pXCrLfYdOWwW4QZxPks5q6j3YgaJpZM4Ko2Ho.

[andeplane]

Ok, I can start thinking about implementing this. I remember a technique where you don't get troubles when adding small quantities to a large cumulative sum.

[rjones-sandia]

the online/realtime algorithm is s bit more complex. give me a minute and I’ll try to dig up the code and a bit of documentation

Reese (Jones) 925 294-4744 rjones@sandia.govmailto:rjones@sandia.gov

On Nov 3, 2016, at 1:19 PM, Anders Hafreager notifications@github.com<mailto:notifications@github.com> wrote:

Ok, I can start thinking about implementing this. I remember a technique where you don't get troubles when adding small quantities to a large cumulative sum.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/lammps/lammps/issues/251#issuecomment-258261924, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AHZBhdnOI5-sB1u-GyhPaplhThi0z7Gsks5q6kHmgaJpZM4Ko2Ho.

[rjones-sandia]

ok I think the algorithm embedded in the attached code (spawned from a now ancient version of fix_ave_time) is basically the one described in :

https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Online_algorithm

I used this extensively to get error estimates of time averages — i.e. I think it works fine. Reese

[andeplane]

Ahh perfect! So am I understanding correctly if you have an old implementation in fix_ave_time which is not in the current version?

Edit: yeah this is the one I was thinking about.

[andeplane]

@rjones-sandia could you send me the code you mentioned directly on email? Github ignores attachments when replying to issues through emails. My email is andershaf@gmail.com.

[akohlmey]

This has been inactive for a while. Are there any updates?

[andeplane]

No updates. I haven't gotten any reply from @rjones-sandia yet and I don't want to reimplement it since most of it is already done.

[andeplane]

@rjones-sandia ping?

[andeplane]

@sjplimp, @stanmoore1, @athomps could you ask @rjones-sandia about this?

[zqex]

This comes with some caveats. While its easy to calculate some number < (x-)^2> for some series of data, this is only the variance, IFF the sampled x'es are statistically independent, which depends on details of the observable and the system.

To estimate the variance, data should be blocked and the convergence after subsequent blocking transformations should be studied. See e.g. the algorithm in Flyvbjerg Pedersen J.Chem.Phys. 1, 461 (1989). http://dx.doi.org/10.1063/1.457480 Not that trivial...