search

rust-random / rand

A Rust library for random number generation.
https://crates.io/crates/rand
Other
1.67k stars 432 forks source link

Cdf testing with Kolmogorov Smirnov #1494

Closed benjamin-lieser closed 1 month ago

benjamin-lieser commented 2 months ago

Summary

As addition and alternative to pdf testing, I implemented distribution testing based on cdf

Motivation

The cdf tests should be more robust and avoid having the bin-width choice problem

Details

I have two simple example tests for normal and binomial. If we like these tests we should also add some way to test on different parameter sets and the other distributions.

This can't be easily generalized to multivariate ones.

benjamin-lieser commented 1 month ago

I am quite happy with the code. After a review I would add the missing tests.

We might have to wait until a release of statrs which can be configured to not depend on rand_distr. Or we get the cdf implementations from elsewhere or have our own. It is only a dev-dep, so the current solution (installing it from their master branch) might be fine

benjamin-lieser commented 1 month ago

I got an interesting test failure on macos:

---- distr::bernoulli::test::bernoulli_distributions_can_be_compared stdout ----
thread 'distr::bernoulli::test::bernoulli_distributions_can_be_compared' panicked at src/distr/bernoulli.rs:229:9:
assertion `left == right` failed
  left: Err(InvalidProbability)
 right: Ok(Bernoulli { p_int: 18446744073709551615 })
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Could be a compiler bug, maybe?

Maybe someone with a mac can reproduce this?

dhardy commented 1 month ago

It passed when re-run.