pynbody / pynbody

N-body and hydro analysis tools for Python 3. (Older versions support Python 2.)
http://pynbody.github.io/pynbody/index.html
161 stars 76 forks source link

Enable smoothing at double precision #795

Closed apontzen closed 4 months ago

apontzen commented 4 months ago

Previously, all smoothing operations took place only with float precision.

Changing this requires very significant surgery on the guts of the KDTree smoothing operation. I have taken the opportunity to reimplement it using an STL heap rather than the old-style C macros. This has actually slowed down the performance somewhat (~20%) and it would be nice to get that performance back. On the other hand the improvement in code clarity and the ability to calculate at higher precision is important.

This shows up in the unit tests, where very low precision was being demanded. When running in parallel, order of evaluation could give floating point errors in the smooth array, which in turn accumulated into fairly significant errors in smoothed quantities. In particular using the Wendland kernel, which is very steep on its outer edge, could accumulate random errors as high as 1% from the forced (and silent) use of float32, which was very confusing.

While debugging this it became clear that the former unit test for the Wendland kernel was actually doing nothing at all, merely loading in an existing density array from disk (!) This has been fixed

In the long term, recovering some of the slow-down by going back to a more efficient priority queue would be very nice, but it should not be at the expense of clarity and flexibility. This will be added to the 'long term kdtree' ticket (#770).