issues
search
Modalities
/
modalities
A framework for training multimodal foundation models.
MIT License
57
stars
5
forks
source link
feat: integrated pytorch swiglu and pytorch flash attention
#209
Closed
mali-git
closed
1 month ago
mali-git
commented
1 month ago
What does this PR do?
This PR ..
General Changes
..
Breaking Changes
..
Checklist before submitting final PR
[ ] My PR is minimal and addresses one issue in isolation
[ ] I have merged the latest version of the target branch into this feature branch
[ ] I have reviewed my own code w.r.t. correct implementation, missing type hints, proper documentation, etc.
[ ] I have run a sample config for model training
[ ] I have checked that all tests run through (
python tests/tests.py
)
What does this PR do?
This PR ..
General Changes
Breaking Changes
Checklist before submitting final PR
python tests/tests.py)