manodeep / Corrfunc

⚡️⚡️⚡️Blazing fast correlation functions on the CPU.
https://corrfunc.readthedocs.io
MIT License
167 stars 51 forks source link

Performance regression for small rmax #65

Closed manodeep closed 8 years ago

manodeep commented 8 years ago

The implementation of assign_ngb_cells is very sub-optimal and requires allocating a totncells^2*8 bytes array. For small rmax, totncells = NLATMAX * NLATMAX * NLATMAX = 10^6. Therefore, the code wants to allocate a 10^12 * 8 ~ 7.5 GiB array. Not only is this wasteful, performance is severely compromised. The assign_ngb_cells takes ~3seconds while the actual pair-counting takes only ~0.5s.

This issue came up while trying to compare to the range_search routine in @mlpack.