Open pfeatherstone opened 1 year ago
Please use torch.nn.functional.scaled_dot_product_attention where possible, e.g., Conformer and Transformer models.
I would like to save on memory and compute.
🚀 Feature request
Please use torch.nn.functional.scaled_dot_product_attention where possible, e.g., Conformer and Transformer models.
Motivation
I would like to save on memory and compute.