haotian-liu / LLaVA

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.
https://llava.hliu.cc
Apache License 2.0
20.42k stars 2.26k forks source link

[Discussion] ms-swift对于Llava系列微调(finetune)的支持 #1304

Open Jintao-Huang opened 8 months ago

Jintao-Huang commented 8 months ago

Discussion

Llava仓库本身已经提供了非常优秀的微调脚本.

ms-swift多模态大模型微调框架集成了Llava的推理与微调, 并书写了最佳实践: https://github.com/modelscope/swift/blob/main/docs/source/Multi-Modal/llava%E6%9C%80%E4%BD%B3%E5%AE%9E%E8%B7%B5.md

如果有感兴趣的小伙伴, 可以来使用😊

The Llava repository itself provides excellent fine-tuning scripts.

The ms-swift multi-modal large model fine-tuning framework integrates Llava's inference and fine-tuning and documents best practices: https://github.com/modelscope/swift/blob/main/docs/source/Multi-Modal/llava%E6%9C%80%E4%BD%B3%E5%AE%9E%E8%B7%B5.md

If any interested individuals, feel free to use it! 😊

wxxu-cnic commented 1 month ago

支持多图微调和推理吗