search

tristandeleu / pytorch-meta

A collection of extensions and data-loaders for few-shot learning & meta-learning in PyTorch
https://tristandeleu.github.io/pytorch-meta/
MIT License
1.97k stars 256 forks source link

Fix/multi head attention #171

Open Clyde21c opened 1 year ago

Clyde21c commented 1 year ago

In the MetaMultiheadAttention class, for a batch_first case, the code corresponding to pre-transpose is missing. Additionally, as the batch_first argument has been used in nn.MultiHeadAttention from PyTorch 1.9, I updated the README.md accordingly.

Refrerence

PyTorch 1.9 PyTorch 2.0

Clyde21c commented 1 year ago

BTW, I sincerely appreciate you for releasing this repository. I published a book about Meta-learning in Korea and utilized the Torchmeta library for the example code for the book.