RickKessler / SNANA

Supernova Analysis package
36 stars 23 forks source link

Use buffer protocol to pass data from Python SED modules #917

Closed hombit closed 2 years ago

hombit commented 2 years ago

SED Python module implementation uses lists to pass numerical data from Python to C, but it looks like the actual python models use numpy internally, so back-and-force transformation numpy array -> python list -> double * happens. It is not a fast thing to do: 1) we create a lot of Python float objects at the first step 2) we slowly access list elements on the second step. For example for 10000 (MXLAM_PySEDMODEL value) element array it takes ~0.4ms, so for simulating a light curve with Nobs=1000 it would spend ~0.4s just doing these transformations:

In [1]: x = np.arange(10000)
In [2]: %timeit np.array(list(x))
390 µs ± 2.85 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

I suggest to use the buffer protocol instead and return numpy arrays from Python methods (or any other objects supporting the protocol) https://docs.python.org/3/c-api/buffer.html

gnarayan commented 2 years ago

Please provide a working example with the AGN model code. The conversion to python list is not ideal, but was reliable and worked with the SNANA C code.