请帮忙解答一下训练的问题

[XA-hyy]

作者你好，我在复现bunny_v1_0-2B-zh的过程中，pretrain阶段训出来保存的config.json中model_type是bunny-qwen，而开源中的config.json的model_type是bunny-qwen2，这个是什么原因？另外请问你在huggingface中开源的configuration_bunny_qwen2.py和modeling_bunny_qwen2.py这两个文件，是做什么用的？训练时是需要把这两个文件替换上去吗？

[Isaachhh]

For the first question, sorry for that we made a mistake here and here, it should be bunny-qwen2. But it's just about name and won't influence the results.

For the second question, https://github.com/BAAI-DCAI/Bunny/issues/39#issuecomment-2021981770

[Isaachhh]

We fix the typo in https://github.com/BAAI-DCAI/Bunny/commit/dcda8a999ef7240619ddca82f19c3054fe54e6dd. Thanks!

[XA-hyy]

感谢回答！第一阶段的命名我这边测试ok了。还有一些问题想请教一下： (1) 二阶段指令微调时，使用finetune_full.py模式，得到的config.json和你开源的bunny_v1_0-2B-zh中开源的config.json差异很大，比如vocab_size我得到的是151936，而你最终的是151646。还有mm_projector_lr我得到null，你的是2e-05；我尝试指令微调时将用bunny_v1_0-2B-zh的配置文件，结果报错mismatch for lm_head.weight [151936, 2048] and [151646, 2048]。请问这个vocab_size、mm_projector_lr等为什么会有差异？ (2) 如何评估第一阶段训练出的pretrain模型的指标？是和二阶段用一样的测试集和测试方法吗？ 感谢！

[Isaachhh]

(1) Actually, you don't need to pay much attention to config.json in Bunny-v1.0-2B-zh, it's the merged weight of bunny-qwen1.5-1.8b-siglip-lora and Qwen/Qwen1.5-1.8B. And the confugurations are inherited from finetune_lora.

for vocab_size, https://github.com/QwenLM/Qwen1.5/issues/29, but I believe you can load and use the model trained by yourself with our source code

for mm_projector_lr, in lora-training, projector is trained under 2e-5 (mm_projector_lr) and the lora adapter of llm is trained under 2e-4 (lr); in full-parameter training, projector and llm are trained under 2e-5 (lr)

(2) Yes. here should support it. But the stage-1 model without visual instruction following may output endlessly.

[Isaachhh]

Close the issue for now if there's no further discussions. Feel free to reopen it if there's any other questions.