ociaw
commented
4 years ago After some quick and dirty tests, using fixed point probabilities does increase performance by ~20% (though this could be due to causing more aggressive pruning than actual speed improvements). However, it's still using the f64 binomial distribution from statrs, so we don't really gain any precision in calculations. We'll need a higher precision binomial pmf, possibly by rewriting the binomial pdf with high-precision fixed point.
Most tests also fail due to the precision problems. However, sub_plane_destroyer goes on indefinitely - I believe this is caused by probability of the generated rounds never quite reaching the prune threshold of zero. We can take a closer look into this once the binomial precision issue is resolved.
Since we're just representing probabilities between 0 and 1 inclusive, we don't need a lot of the features of floating point numbers. We don't need to worry about numbers greater than 1 or less than 0. Additionally, very small probabilities would be pruned out in most instances anyway. The
U1F63type from fixed seems to fit this use case very well. Potential benefits include better performance (since we're dealing with integers) and arguably better precision.