Open prutschman-iv opened 1 month ago
After quite a bit of debugging, it dawned on me that numpy's default dtype is float64, whereas pytorch's default is of course float32. Admittedly this is quite a big oversight on my part!
To confirm, I manually converted all (relevant) internal tensors to torch.float64
, and indeed the precision is now in a reasonable margin of scipy's.
Perhaps I should include an optional argument that forces torchrbf to use higher precision internally -- in most cases it is probably not necessary, so the default can be set to the current behavior (float32). And of course GPU performance tends to fall drastically when using higher precision so you'll want to avoid it when possible.
If you want to quickly hack together a local fix, you can CTRL + F
".float()" in torchrbf/RBFInterpolator.py and change those all to ".double()". You'll also want to make sure your input tensors (data coordinates and data values) are converted.
I'm currently a bit occupied so I won't have time to push a patch for now.
Thank you, I will give your suggestion a try!
I was seeing what seemed like poor numerical performance (without TF32) compared to the scipy implementation of RBFInterpolator, so I made the following test case. It creates a regularly spaced grid of points, displaces them randomly but not so much as to create "pinches", then compares the interpolated values at the regular grid points. With infinite precision a thin plate spline interpolator would return identical values.
When I run this on my Windows machine with a GTX 4090, I get the following:
Do you have any suggestions or advice on improving the situation? 16k control points is a bit on the excessive side, but I have a practical application that could easily use on the order of 4k points, and torchrbf errors at this level are on the order of the grid displacement.