search

hugohrban / ProGen2-finetuning

Finetuning ProGen2 protein language model for generation of protein sequences from selected protein families.
BSD 3-Clause "New" or "Revised" License
24 stars 2 forks source link

Question: Can LoRA be Used to Reduce Resource Requirements for Fine-Tuning? #1

Open liu12347 opened 1 month ago

liu12347 commented 1 month ago

Thank you for providing such a great resources! I was wondering if it is possible to use LoRA (Low-Rank Adaptation) to reduce the resource requirements and potentially fine-tune even larger models. Could you please let me know if LoRA can be integrated with this model?

hugohrban commented 1 month ago

Thanks for your question @liu12347 ! I have not tried using LoRA, but I am pretty sure it should be possible to do it. I uploaded some larger ProGen2 model checkpoints to my hugging face, so you can try using LoRA with them :)