search

balbasty / torch-interpol

High-order spline interpolation in PyTorch
MIT License
62 stars 4 forks source link

Potential bug in sampling single pixel from with grid_pull? #5

Closed Skylion007 closed 1 year ago

Skylion007 commented 1 year ago

Hello,

I wanted to use this library for doing tricubic sampling from a voxel grid. I have tested it with trilinear interpolation and it works fine, however, I get weird errors whenever I try increase the interpolation degree past 1.

In this case, img consists of batches of 4 by 4 by 4 cube of points stored in two channels. "weights" consists of single points we would like to sample from that batch.

Using this library for trilinear sampling works perfectly: 

img = img.reshape(-1,2,4,4,4)
c = interpol.grid_pull(img, weights.view(-1, 1, 1, 1, 3), interpolation=1, prefilter=True)

but when I try to use to increase the interpolation to a higher order, it no longer works. I get this error in nd.py:

        c = interpol.grid_pull(img, weights.view(-1, 1, 1, 1, 3), interpolation=3, prefilter=True)
        ...
        idx = [c[n] for c, n in zip(coords, nodes)]
        idx = sub2ind_list(idx, shape)
        idx = idx.expand([batch, channel, idx.shape[-1]])
              ~~~~~~~~~~ <--- HERE
        out1 = inp.gather(-1, idx)
RuntimeError: The expanded size of the tensor (2) must match the existing size (65536) at non-singleton dimension 1.  Target sizes: [65536, 2, 1].  Tensor sizes: [65536, 1]

Am I misunderstanding something about how to call this library for voxel representations or how the interpolation parameter is used? Is actually doing grid_sample possible with higher order interpolations in this library? Is this a bug caused by accidental squeezing/broadcasting?

balbasty commented 1 year ago

Hi

Thanks for the report! There was indeed a bug in the way I wrote the broadcasting.

I've just pushed a fix to the "fix-issue-5" branch. It does not throw errors anymore, although you may want to make sure that the sampled values make sense (they used to make sense when the batch dimension was 1, which did not trigger the broadcast bug). Would you mind trying out that branch before I merge it?

Cheers Yael

Skylion007 commented 1 year ago

That seemed to fix the error at least. 💯 Thanks for the fast response.