search

modelscope / ms-swift

Use PEFT or Full-parameter to finetune 400+ LLMs or 100+ MLLMs. (LLM: Qwen2.5, Llama3.2, GLM4, Internlm2.5, Yi1.5, Mistral, Baichuan2, DeepSeek, Gemma2, ...; MLLM: Qwen2-VL, Qwen2-Audio, Llama3.2-Vision, Llava, InternVL2, MiniCPM-V-2.6, GLM4v, Xcomposer2.5, Yi-VL, DeepSeek-VL, Phi3.5-Vision, ...)
https://swift.readthedocs.io/zh-cn/latest/Instruction/index.html
Apache License 2.0
4.25k stars 376 forks source link

qwen2 audio微调提问 #2096

Closed zhangfan-algo closed 5 days ago

zhangfan-algo commented 1 month ago

想问下如何在微调的过程中设置只微调语言模型部分，把audio encoder部分屏蔽掉

Jintao-Huang commented 1 month ago

全参数嘛

--freeze_vit true 试试

zhangfan-algo commented 1 month ago

是的 不知道qwen2 audio适配不