search

numba / numba

NumPy aware dynamic Python compiler using LLVM
https://numba.pydata.org/
BSD 2-Clause "Simplified" License
9.65k stars 1.11k forks source link

Faster Mersenne Twister implementation? #3102

Open seibert opened 6 years ago

seibert commented 6 years ago

Given the popularity of Numba for Monte Carlo, we might want to consider using a faster MT implementation than NumPy, since we have to put an implementation into Numba anyway.

Julia uses this: http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/SFMT/#dSFMT

Which seems to give much improved performance over NumPy: http://numpy-discussion.10968.n7.nabble.com/speed-of-random-number-generator-compared-to-Julia-td44136.html

There is also this project: https://github.com/bashtage/randomgen

seibert commented 6 years ago

One problem, of course, is that this would likely break the exact match of RNG output between Numba and NumPy, which was recently achieved by #3038.

synapticarbors commented 6 years ago

Would it be feasible to default to the numpy version of the prng, but package a faster version within numba that could be called as something like nb.random.<func>? That way there could be the consistency with numpy if needed, but a user could make a conscience decision to select a faster implementation with the knowledge that they could no longer exactly match up with an implementation that used numpy.

seibert commented 6 years ago

I'm not sure what the status or roadmap for randomgen is, but if it had a C API, we could make it easy to use from Numba.

dhirschfeld commented 6 years ago

I believe the eventual goal is for randomgen to be incorporated into numpy itself as a new RandomGenerator class.

IIUC the existing RansomState api will remain around indefinitely, mostly for legacy unit-testing purposes though users will be encouraged to use the new api.

https://github.com/numpy/numpy/blob/master/doc/neps/nep-0019-rng-policy.rst#implementation