Bootstrapping, subsample quantity.

[jbrezmorf]

Reimplement Quantity.subsample:

• make it operating on individual chunks without creating a list of selected samples before the chunk processing.
• avoid passing special arguments

Target usage is bootstrapping.

This makes #35 deprecated.

[martinspetlik]

I would wait until we deal with all the changes in othe pull requests, there are definitely changes that affect Quantity.subsample

[jbrezmorf]

Online subsampling algorithms discussed by Vitter

Suppose selection n items from N. The proposed online subsampling algorithm is correct and first proposed by Knuth, that call it S sampling with N random numbers. The improved version is A sampling, with $n$ random numbers.

However we may want to use numpy.random.choice optimized implementation (source). That means: choice(chunk, size = n/N*len(chunk) and then actualize n and N.

[jbrezmorf]

It seems (according experiments) that chunked version of the S-algorithm doesn't work since probability is not correctly updated through the chunk. But for limit cases with chunk size 1 or N it works. Size must be a random variable with suitable distribution.

[jbrezmorf]

Chunk subset size passed as an argument to the choice function should be a random variable with the hypergeometric distribution. The discrete variable k is the size of the chunk subset, parameters are: N - total number of samples, K - total size of subset, n - the chunk size.