lucidrains / x-transformers

A simple but complete full-attention transformer with a set of promising experimental features from various papers
MIT License
4.42k stars 377 forks source link

how to set inputs to the right shape #218

Open emadkavousi opened 7 months ago

emadkavousi commented 7 months ago

Hi and thanks for this great work . I am new with deep learning and i want to use x_transformer in my project but i cant set my inputs in to the right shape.in all examples in github page input shape is like : src = torch.randint(0, 256, (1, 1024)) but when I want to reshape my tensors with reshape or view function of pytorch i get an error for example I want to have input with shape new_shape = (0, 256, (128, 34)) x = x.view(new_shape) but i get this error: x = x.view(new_shape) TypeError: view(): argument 'size' must be tuple of SymInts, but found element of type tuple at pos 3 and no matter how I try and search I cant have this shape for input . is there any easy way to give x-transformer inputs with right shape? am I worng about input shape ? can you explain to me what is these numbers in src = torch.randint(0, 256, (1, 1024)) in input shape ? please help me .

RyanKim17920 commented 6 months ago

The code "src = torch.randint(0, 256, (1, 1024))" creates a torch shape of (1, 1024) with randomized integers from 0 to 255. Trying to do a view causes an error as you are supposed to input a non-proper shape for a vector, causing the related issues.

You can test this by using print(src.shape) which would give you (1,1024).

Please refer to the website for further details: https://pytorch.org/docs/stable/generated/torch.randint.html