Open lucyknada opened 5 months ago
Can you verify that flash attention is installed?
# pip show flash-attn
Name: flash-attn
Version: 2.5.5
Summary: Flash Attention: Fast and Memory-Efficient Exact Attention
Home-page: https://github.com/Dao-AILab/flash-attention
Author: Tri Dao
Author-email: trid@cs.stanford.edu
License:
Location: /root/miniconda3/envs/py3.11/lib/python3.11/site-packages
Requires: einops, ninja, packaging, torch
Required-by:
seems like it (inside the docker image)
Please check that this issue hasn't been reported before.
Current behaviour
crashes with:
Steps to reproduce
run that docker, modify the mistral qlora example
(no matter the options, it'll still crash)
Config yaml
Possible solution
No response
Which Operating Systems are you using?
Python Version
3.10, 3.11
axolotl branch-commit
9b6ee83a73d5ffbdc33cfb383a131a08c2b594ff
Acknowledgements