Tensor unfold?

[grzuy]

Hi there,

Any tips on how would one could achieve something similar/close to pytorch's https://pytorch.org/docs/stable/generated/torch.Tensor.unfold.html?

Something built on top of narrow maybe?

Thanks!

[EricLBuehler]

If you know the number of dims of the input tensor, perhaps you can do the following:

• Call a .to_vecX() method
• Call something like .windows to actually do the "unfolding"
• Then, convert that data back This doesn't actually unfold in a specified dimension, but you can do that by calling windows on a specific dimension of the .to_vecX() return value. Would this work?

[mokeyish]

You can try to implement unfold in this candle-ext

[mokeyish]

It seems that unfold calculates a new shape and then reshapes it? It may also need permute.

[LaurentMazare]

You should avoid doing to_vec for such ops as much as possible, this breaks backprop so training isn't possible when using it and has some large overhead when running on gpus/accelerated devices (and even on cpu). Instead you probably want to generate a tensor with the indexes of the target positions based on the unfold parameters and call gather or some equivalent function on the source tensor using these indexes.

[grzuy]

Thanks all for the feedback :+1:

[EricLBuehler]

@grzuy, did you have success implementing this?

[grzuy]

@grzuy, did you have success implementing this?

Hi @EricLBuehler , Didn't try in the end.

[EricLBuehler]

@grzuy I actually just wrote some code to do it on the CPU in the end.