search

openspeech-team / openspeech

Open-Source Toolkit for End-to-End Speech Recognition leveraging PyTorch-Lightning and Hydra.
https://openspeech-team.github.io/openspeech/
MIT License
670 stars 112 forks source link

torch.nn.functional.scaled_dot_product_attention() #199

Open pfeatherstone opened 1 year ago

pfeatherstone commented 1 year ago

🚀 Feature request

Please use torch.nn.functional.scaled_dot_product_attention where possible, e.g., Conformer and Transformer models.

Motivation

I would like to save on memory and compute.