search

BAAI-DCAI / Bunny

A family of lightweight multimodal models.
Apache License 2.0
894 stars 67 forks source link

请帮忙解答full-parameter finetune的一些问题 #69

Closed Why0912 closed 4 months ago

Why0912 commented 5 months ago

在第二阶段使用phi-2和siglip进行全参数微调得到的checkpoint文件与Readme中给出的[Bunny-v1.0-3B/bunny-phi-2-siglip]相比较config.json中缺少“auto_map”这一项，在补全这一项以及两个关联的py文件之后可以正常运行。 这是什么原因造成的？为什么全参数微调得到的checkpoint文件会缺少？

Isaachhh commented 5 months ago

Bunny-v1.0-3B is trained by LoRA and we merge the weights.

The snippet in Quickstart is used for Bunny-v1.0-3B (SigLIP + Phi-2) and so on. We manually combine some configuration code into a single file for users' convenience. Also, you can check modeling_bunny_phi.py and configuration_bunny_phi.py and their related parts in the source code of Bunny to see the difference.

For other models including models trained by yourself, we currently only support loading them with installing source code of Bunny. Or you can copy modeling_bunny_phi.py and configuration_bunny_phi.py into your model and edit config.json.

Isaachhh commented 4 months ago

Close the issue for now if there's no further discussions. Feel free to reopen it if there's any other questions.