search

THUDM / CogVLM

a state-of-the-art-level open visual language model | 多模态预训练模型
Apache License 2.0
5.92k stars 406 forks source link

请问换成其他LLM加载模型权重的时候，hf版本能和sat适配吗，还是说要先把hf的权重转成sat的 #366

Closed AlenjandroWang closed 5 months ago

1049451037 commented 8 months ago

sat里支持很多常见的LLM：https://github.com/THUDM/SwissArmyTransformer/tree/main/examples

AlenjandroWang commented 8 months ago

sat里支持很多常见的LLM：https://github.com/THUDM/SwissArmyTransformer/tree/main/examples

[transform_param.py]是可以把hf权重转换成sat吗

1049451037 commented 8 months ago

是的

AlenjandroWang commented 8 months ago

Dear author

I'm an undergraduate student majoring in Computer Science from the Northwestern Polytechnical University. I recently worked on MLLM research and read your paper "COGVLM: VISUAL EXPERT FOR LARGE LANGUAGEMODELS", I am very interested in and admire your work. Your paper has provided me with a lot of inspiration and enlightenment ，so could you please share your pretraining code with me, for research purposes only? I promise that if I end up using your code, I will cite your work correctly in my paper. Thank you very much.

Sincerely, Dianyi

---原始邮件--- 发件人: "Qingsong @.> 发送时间: 2024年2月4日(周日) 中午12:46 收件人: @.>; 抄送: "Alex @.**@.>; 主题: Re: [THUDM/CogVLM] 请问换成其他LLM加载模型权重的时候，hf版本能和sat适配吗，还是说要先把hf的权重转成sat的 (Issue #366)

是的

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

ayensujeremiah commented 7 months ago

Dear authors, I am conducting a research around multimodal large language models, could you share how image and text inputs are encoded or preprocess for the Supervised finetuning as you did with LLAVA instruct? I promise to cite your work correctly in my paper as well