search

sdtaylor / phenology_forecasts

The backend for http://phenology.naturecast.org
http://phenology.naturecast.org
8 stars 1 forks source link

large bottleneck in np.exp #33

Closed sdtaylor closed 6 years ago

sdtaylor commented 6 years ago

profiling shows np.exp() to be taking quit a long time. in the forecast runs.

Testing shows the following on serenity (ubuntu 16.04, conda install python3.6)

shawn@serenity:~$ python
Python 3.6.3 | packaged by conda-forge | (default, Nov  4 2017, 10:10:56) numpy 1.15
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import timeit
>>> timeit.timeit('np.exp(d)', setup='import numpy as np;d=np.ones((420,1405,620))',number=1)
27.85343541414477

A windows machine in the library runs the same command in 1.5 seconds.

This issue potentially points to a glibc issue .

sdtaylor commented 6 years ago

potentially fixed by switching over to np.float32. see https://github.com/sdtaylor/pyPhenology/pull/71

sdtaylor commented 6 years ago

getting around this with more brute force parallel processing