search

rust-random / rand

A Rust library for random number generation.
https://crates.io/crates/rand
Other
1.67k stars 432 forks source link

Loss of precision in Weibull #1507

Closed benjamin-lieser closed 1 month ago

benjamin-lieser commented 1 month ago

Weibull::new(1000.0, 0.001) will sample inf a decent number of times. The expectation of 4.023873e+2570 can of course not be sampled. The median of 6.7e-160 is reproduced well thought.

Do we want to allow this extreme values? At least we should document the limits of the implementation/floating point

dhardy commented 1 month ago

It sounds like the algorithm is working correctly then, albeit the KS statistic is unable to test this where a significant portion of outputs are inf?

Unless we have grounds to believe an output is wrong (besides +/- inf) then I don't think we should disallow those parameters. (We could document which parameters were tested and/or which parameters might be expected to yield unrepresentable values.)

benjamin-lieser commented 1 month ago

I am pretty sure the output is correct, expect for the under and overflows. 0.0 is also more sampled than it should because of underflow, this makes the KS test fail. The same thing probably happens at the other end of the distribution with inf.

I would document the bounds at which a significant amount of samples will be the result of under or overflow.

dhardy commented 1 month ago

Underflow to zero and subnormal samples can influence the shape of the CDF, thus affecting the test. It seems unlikely to be an issue in usage however; anyone working with very small outputs ought to already have an idea of the limits of FP precision.