Closed rotameraklisi closed 3 years ago
Hi, I'm sorry but I can't really help you on these dimension errors, it would require that I run the entire code with your dataset, which would take more time that I can afford. Especially considering that your using a slightly modified version of my code, such as:
self._scores = (queries@keys.T) / np.sqrt(K)
This does make the code much more readable, and follows the PEP, but I didn't had a chance to test it thoroughly.
I would encourage you to continue your debugging to understand why the batch matrix multiplication raises an error. You could find it helpful to start with the original code, and a fresh dataset, to get a working example, and then work back to your particular case. I would also encourage you to take a look at the equations behind the Multi Head Attention, in order to get a better understanding on how things should be multiplied.
Hello, I'm working on time series classification with transformer. I divide it into 10 classes with 14 features, 1 label (categoric-> LabelEnceder) d_input = 14, d_outpu = 10, d_model? Window_size = 16, but I am getting such an error. What should I do, why does the dimension of y appear one dimension?
[Epoch 1/2]: 0%| | 0/31514 [00:00<?, ?it/s]torch.Size([31514, 16, 14]) torch.Size([31514]) torch.Size([6092, 16, 14]) torch.Size([6092]) 31514 6092 Running on the GPU Using device cuda:0 torch.Size([64, 16, 14]) torch.Size([64]) [Epoch 1/2]: 0%| | 0/31514 [00:00<?, ?it/s]torch.Size([896, 16, 14]) torch.Size([896, 16, 14])
error------
RuntimeError Traceback (most recent call last)