darach / eep-js

eep.js - Embedded Event Processing
MIT License
217 stars 33 forks source link

Numerically stable aggregates #2

Closed redchair123 closed 12 years ago

redchair123 commented 12 years ago

The most numerically stable way to take a rolling average is to use the strategy

new_average =  old_average + (new_value - old_average) / new_count

MeanFunction's accumulate should be

self.accumulate = function(v) { s+=(v-s)/++n; }
self.emit = function() { return s; }

Likewise, variance can use the form

Var(X) = E[X^2]-E[X]^2

where both are simple means, admitting a similar solution to above (e2 +=(v*v - e2)/n for second moment)

darach commented 12 years ago

Confirmed. Fixed in a slightly different way in 0.2.