Closed honnibal closed 8 years ago
Hi,
Not sure I totally see at what level this API would exist. Would it hang off of a RandomState
instance? I suppose it needs to avoid directly handling the actual state.
I also don't quite understand the function name - why _from_normal
?
Presumably you would want a simple wrapper of the function random_uniform_fill
in distributions.c
(https://github.com/bashtage/ng-numpy-randomstate/blob/master/randomstate/distributions.c#L39 ).
I'm still not totally sure how you can completely avoid GIL since using a RandomState
instance requires accessing self
, which requires GIL
.
Sorry I should've been a bit clearer. By normal
I meant, draw from a normal distribution. I thought I'd said above that I need to draw from a Gaussian, but I see that I didn't.
You can read from and write to self
attributes without the GIL, so long as they're cdef
attributes. You can't access self.foo
if foo
is a Python object, though. I guess I should've spent more time understanding the design. I'll take another look.
Hmm. You could have a cdef
method of RandomState
that made a call to random_gauss_fill
. But that actually isn't so helpful. In the Python version, you have the state object as a global variable, and you just add these methods to the global namespace by assigning them to global variables.
In Cython this wouldn't work, so you'd only be able to use these cdef methods if you first create a new RandomState instance, or pass one in. But in both cases, you'd have to acquire the GIL.
Maybe have a cdef
function that did the setup and teardown around a call to random_uniform_fill
?
I would suppose the simplest method would be to write a basic functional interface that would have signatures like
cdef seed(aug_state* state) nogil:
# Do seeing stuff
cdef normals(aug_state* state, double* out, int n) nogil:
random_normal_fill(state, out, n)
The only difficulty with this is that the structure aug_state
isn't very friendly.
Some of the PRNGs are easier to use than others -- in particular xorshift
use arrays of uint64
so a basic state could be easily manipulated using only NumPy arrays (or directly using malloc).This isn't really the same as an aug_state which has place holders for lots of other stuff that isn't needed for most distributions.
I guess I really only need xorshift + Ziggurat. So maybe I should just extract the things I need and make my own little package of them.
I was tempted to say the state can just live as a global variable. But if there are race conditions that make the random sequence unpredictable at unpredictable times, eventually I'll probably go crazy debugging something. So I should probably just accept some set-up/tear down.
Thanks for the help.
I think you are right, that in the special case where MT code needs to release the GIL a lot of care is needed, and so it is probably easiest to use xorshift1024 + splitmix64 for seeding directly.
Hi,
I need to repeatedly create a vector of a around 10^6 random doubles, as part of some neural network code I'm writing in Cython. I want to release the GIL around this function, so I need:
1) A fast PRNG; 2) With a permissive license; 3) With a public C-level API.
It looks like you've got 1 and 2, so I'd like to see whether I can add 3 :). For my purposes, the ideal API would be something like this:
I understand that the goal of this repository is to get integrated into numpy. But, would you accept a pull request with a Cython .pxd file, the
nogil
functions, and the appropriate changes to the setup.py?