您好，是否可以支持CogVLM等多模态模型？

GraphPKU / PiSSA

PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models(NeurIPS 2024 Spotlight)

https://arxiv.org/abs/2404.02948

261 stars 9 forks source link

Closed Shubao-Zhao closed 4 months ago

fxmeng commented 4 months ago

我们目前除了LLM中的线性层，还支持了卷积层和embedding层。有了这些模块，多模态模型也可以用上PiSSA。我们将会初始化并分享相应初始化的checkpoint，敬请期待。

Shubao-Zhao commented 3 months ago

期待您分享一个用PiSSA在多模态模型(如CogVLM、InternVL等)微调的案例