hiyouga / LLaMA-Factory

Unify Efficient Fine-Tuning of 100+ LLMs

Apache License 2.0

25.26k stars 3.13k forks source link

support flash-attn in Dockerfile #4461

Closed hzhaoy closed 1 day ago

hzhaoy commented 3 days ago

What does this PR do?

flash-attn is mandatory for some models. Uninstalling and then rebuilding flash-attn (following the installation) satisfied the requirement and fixed #4242 #4264 completely.

Before submitting

[x] Did you read the contributor guideline?
[x] Did you write any new necessary tests?

hiyouga commented 2 days ago

Please take into consideration the GPU that does not support flash attn https://github.com/hiyouga/LLaMA-Factory/blob/8d6cd69ac43afd4bd7c14bd02b0061455827ac9e/docker/docker-cuda/Dockerfile#L8