search

Alpha-VLLM / LLaMA2-Accessory

An Open-source Toolkit for LLM Development
https://llama2-accessory.readthedocs.io/
Other
2.68k stars 170 forks source link

The stage_2 checkpoint include QFormer part? #74

Open WeiXuanLi-1024 opened 11 months ago

WeiXuanLi-1024 commented 11 months ago

Does this tage_2 checkpoint weights your provide include Qformer part weights which PEFT from data {alpaca_gpt4_data.json and lava_instruct_150k.json } ?
https://huggingface.co/Alpha-VLLM/LLaMA2-Accessory/tree/main/finetune/mm/alpacaLlava_llamaQformerv2Peft_13b

ChrisLiu6 commented 11 months ago

抱歉没太能够理解您的问题。[https://huggingface.co/Alpha-VLLM/LLaMA2-Accessory/tree/main/finetune/mm/alpacaLlava_llamaQformerv2Peft_13b]()是基于[https://huggingface.co/Alpha-VLLM/LLaMA2-Accessory/tree/main/finetune/mm/caption_llamaQformerv2_13b]()，再在alpaca和llava上训练得到的，包含推断所需的所有参数

WeiXuanLi-1024 commented 11 months ago

你们在训练stage_2的时候，训练出来两个权重 alpacaLlava_llamaQformerv2Peft_13b 和 alpacaLlava_llamaQformerv2_13b ，前者是部分PEFT,后者是全参数微调, 我想请教的是，在训练过程中，PEFT只有llama参与训练了吗？全参数微调Qformer 部分有参与更新吗？

ChrisLiu6 commented 11 months ago

目前，qfromer内部的参数在PEFT和全参数微调中都是没有被更新的、