search

ggerganov / ggml

Tensor library for machine learning
MIT License
11.22k stars 1.04k forks source link

Possible bug in ggml_permute #957

Closed bernerprzemek closed 2 months ago

bernerprzemek commented 2 months ago

I found this issue, when I permutate tensor without changing axis: struct ggml_tensor * t = ggml_new_tensor_4d(ctx, type, 4, 3, 1, 1); struct ggml_tensor * resultt = ggml_permute(ctx, t, 0, 1, 2, 3); result is as expected: nb = {4, 16, 48, 48} but doing this: struct ggml_tensor * resultt = ggml_permute(ctx, t, 0, 1, 2, 3); result is: nb = {16, 4, 48, 48} I'm not very familiar with gglm but in that case type of tensor is no longer float because nb[0]!=sizeof(float), and further operations giving assertion error like: ggml.c:10636: GGML_ASSERT(src0->nb[0] == sizeof(float)) failed

JohannesGaessler commented 2 months ago

nb is the stride between elements in bytes. ggml_permute changes the ne and nb values in such a way that the order in which data is iterated over is changed without touching the actual data. nb[0] == sizeof(float) means that the tensor is contiguous in memory in its first dimension which is needed for some operations. This has nothing to do with the data type and is only due to non-contiguous input tensors not being supported. You should be able to fix the problem by inserting ggml_cont after ggml_permute.

bernerprzemek commented 2 months ago

Thanks somehow I missed this part (inserting ggml_cont)