test-time-training / ttt-lm-pytorch

Official PyTorch implementation of Learning to (Learn at Test Time): RNNs with Expressive Hidden States
MIT License
1.01k stars 56 forks source link

The multiplication of matrix #23

Closed Z-Z188 closed 1 month ago

Z-Z188 commented 1 month ago

image Hello author, thank you very much for your work. But I have a problem with the multiplication of this matrix: if X is a matrix of size (d,b), then the transpose of X x X has size (b, b), and WX-X has size (d, b), then how can the two matrices be multiplied? Look forward to your reply, thank you!

karan-dalal commented 1 month ago

This was a typo in an earlier submission. The updated equation is on Arxiv. https://arxiv.org/pdf/2407.04620

Screenshot 2024-09-08 at 12 02 57 AM
Z-Z188 commented 1 month ago

This was a typo in an earlier submission. The updated equation is on Arxiv. https://arxiv.org/pdf/2407.04620 Screenshot 2024-09-08 at 12 02 57 AM

Thanks!