search

cmnfriend / O-LoRA

MIT License
126 stars 12 forks source link

Could this be adapted to Llama2? Which part should be modified? #2

Open Labmem009 opened 8 months ago

Labmem009 commented 8 months ago

Great work! This repo is built based on the pretrained T5-large model, but I wonder if it could be adapted to Llama2? And which part of this project should be modified? Thanks a lot!

Labmem009 commented 8 months ago

I noticed the llama branch, which part of this branch should be modified to be adapted to Llama2? Or maybe the llama branch is not an implementation of O-LoRA? And I noticed that in your paper you trained another version of O-LoRA for Llama-7B, could you please release this version or a version adapted to Llama2?

cmnfriend commented 8 months ago

Sorry for late response, Maho senpai. Now the code has been updated to adapt LLaMA2.