Some ideas and links - Githubissues

mozilla / measure-noise

Measure how our data deviates from normal distribution

Mozilla Public License 2.0

11 stars 10 forks source link

Some ideas and links #5

Open klahnakoski opened 5 years ago

klahnakoski commented 5 years ago

The general challenge is arriving at a reliable statistic given how few data points we have. A statistic that measures deviation-from-Gaussian may be the best.

Start with measuring how well the model fits https://en.wikipedia.org/wiki/Likelihood-ratio_test

some sort of power analysis to verify how this may fail https://en.wikipedia.org/wiki/Power_(statistics)

help selecting a model https://en.wikipedia.org/wiki/Bayes_factor

using bayes to select a model, including accounting for free variables https://en.wikipedia.org/wiki/Bayes_factor

we are reducing type 1 errors (the false positives) https://en.wikipedia.org/wiki/Type_I_and_type_II_errors#Type_I_error

more on model selection https://en.wikipedia.org/wiki/Model_selection

should we account for absolute value? standard deviation wrt mean
distinguish between outliers and rare modes
the replicates may help indicate what the mean represents
can we ignore skew and focus on kurtosis?

see applications at bottom, where goodness of fit is linked https://en.wikipedia.org/wiki/Kurtosis

2014 paper on kurtosis measure is propensity for outliers https://en.wikipedia.org/wiki/Kurtosis#cite_note-4

https://en.wikipedia.org/wiki/Normality_test

klahnakoski commented 5 years ago

Jarque–Bera test https://en.wikipedia.org/wiki/Jarque%E2%80%93Bera_test