Substitution/scaled dot product attention - Githubissues

sony / model_optimization

Model Compression Toolkit (MCT) is an open source project for neural network model optimization under efficient, constrained hardware. This project provides researchers, developers, and engineers advanced quantization and compression tools for deploying state-of-the-art neural networks.

https://sony.github.io/model_optimization/

Apache License 2.0

331 stars 53 forks source link

Substitution/scaled dot product attention #1229

Closed yarden-yagil-sony closed 1 month ago

yarden-yagil-sony commented 2 months ago

Pull Request Description:

Checklist before requesting a review:

[x] I set the appropriate labels on the pull request.
[x] I have added/updated the release note draft (if necessary).
[x] I have updated the documentation to reflect my changes (if necessary).
[x] All function and files are well documented.
[x] All function and classes have type hints.
[x] There is a licenses in all file.
[x] The function and variable names are informative.
[x] I have checked for code duplications.
[x] I have added new unittest (if necessary).