stan-dev / stan

Stan development repository. The master branch contains the current release. The develop branch contains the latest stable development. See the Developer Process Wiki for details.
https://mc-stan.org
BSD 3-Clause "New" or "Revised" License
2.57k stars 368 forks source link

Experiment: Switch RNG from ecuyer1988 to mixmax #3169

Closed WardBrian closed 1 year ago

WardBrian commented 1 year ago

Submission Checklist

Summary

Experiment to see how viable it is to change RNGs, see #3167 and discussion.

Also needs https://github.com/stan-dev/stanc3/tree/experiment/mixmax-rng

Intended Effect

How to Verify

Side Effects

Documentation

Copyright and Licensing

Please list the copyright holder for the work you are submitting (this will be you or your assignee, such as a university or company): Simons Foundation

By submitting this pull request, the copyright holder is agreeing to license the submitted work under the following licenses:

WardBrian commented 1 year ago

This seems fine up to the performance tests, which don't yet pass on the stanc3_bin_url parameter so can't be run yet.

I had to revert some of the tests to using ecuyer1988 in e026895 because a fair number of tests here seem to have "magic" seeds and assert on certain values, and I cannot tell if those values were derived somehow of if the existing output was just frozen in the tests

WardBrian commented 1 year ago

Interesting: Up at the performance-tests-cmdstan level, this change had two models fail https://jenkins.flatironinstitute.org/blue/organizations/jenkins/Stan%2FCmdStan%20Performance%20Tests/detail/downstream_tests/391/pipeline

One of them is a data generation model which heavily uses RNGs, so it is expected that it would likely change. The other is low_dim_gauss_mix_collapse (here)

WardBrian commented 1 year ago

Closing for now, if we ever wanted to make this change it seems like we'd just need to update the magic numbers in a few of our tests