search

GraphPKU / PiSSA

PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models(NeurIPS 2024 Spotlight)
https://arxiv.org/abs/2404.02948
261 stars 9 forks source link

reproduce the results #4

Closed lucasliunju closed 6 months ago

lucasliunju commented 6 months ago

Thanks for your great work.

I would like to ask whether we can use this codebase to reproduce the results about lora anf full fine-tuning. The main reason is that I would like to fairly comapre the performance between pissa, lora anf full fine-tuning.

Best

fxmeng commented 6 months ago

Thank you for your interest in PiSSA. It takes some time to clean up the codes and training scripts, which are now available at: https://github.com/GraphPKU/PiSSA/tree/main/scripts. You can use different scripts from this directory according to your needs.

lucasliunju commented 6 months ago

Thank you very much for your answer and great work!