Write up the docs for experiments

[sergey-a-berezin]

The top-level README.md should, of course, describe the basic usage, but also be the entry point to the detailed description of each of the experiments.

Perhaps, a good practice is to keep the detailed write-up of each experiment in its own package together with the code. This would naturally organize the descriptions, and the top-level can then provide the table of contents with the suggested order of reading.

[sergey-a-berezin]

Saving a paragraph for later:

We will also attempt approximating the sum of N t-distributed samples with either another t-distribution with a different a, or a normal distribution. For the latter, it would be interesting to find such N when such approximation could be justified.

[sergey-a-berezin]

Quick plan for the power distribution experiment writeup

• Finish mean, MAD, sigma and alpha for the t-distribution; in particular, compare the source's tails with normal
• Introduce a "cheat": cumulative plots to estimate CI with a single large sample
• Plot cumulative skewness and kurtosis for the normal and t-distribution

[sergey-a-berezin]

Add a top-level section for the most important results, such as the kind of distribution, the precision of parameters, etc., with links to the detailed sections.

[sergey-a-berezin]

• Split long sections into separate files, link to them from the sections' README's.
• Also create subsections in the top-level README file.
• Add "next" and "previous" links to the top and bottom of each file, for a better continuous reading experience.

[sergey-a-berezin]

Optimization ideas:

• Fast compounding, and how it compares with direct compounding for (a) the same number of source samples, (b) the same number of compounded samples
• Approximation of compounding with a sample distribution (sample source N times, use these samples for constructing a compounded distribution)
• Compounding based on biased sampling, a.k.a. Monte Carlo integration; 2-step compounding (e.g. 252 = 6 x 42), and sampling from the resulting histogram

[sergey-a-berezin]

In Hypothesis testing, fix the definition of a confidence interval and p-value. CI is a pair of statistics, and p-value is the probability of the experimental result based on the hypothesized distribution. In fact, don't use p-values, they are not very useful. Try to stick with the probability of the hypothesis being true whenever possible.