search

zjysteven / lmms-finetune

A minimal codebase for finetuning large multimodal models, supporting llava-1.5/1.6, llava-interleave, llava-next-video, qwen-vl, phi3-v etc.
Apache License 2.0
127 stars 9 forks source link

V100 support ? #37

Open hfutzzw opened 2 days ago

hfutzzw commented 2 days ago

请问微调训练支持V100服务器吗，8*32G？V100原生不支持Flash Attention？

zjysteven commented 2 days ago

我们对flash attention的使用与huggingface trainer一致，可以通过参数disable