search

huggingface / peft

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.
https://huggingface.co/docs/peft
Apache License 2.0
14.86k stars 1.41k forks source link

Tutorial notebook for applying PEFT with DNA Language models #1837

Open rahulbshrestha opened 3 weeks ago

rahulbshrestha commented 3 weeks ago

Feature request

A notebook tutorial that shows an example of how to use parameter-efficient fine-tuning techniques from the PEFT library (e.g LoRA, IA3, etc.) with a DNA Language Model (e.g https://huggingface.co/gagneurlab/SpeciesLM).

The tutorial will contain an example of how to load and apply a DNA language model with and without PEFTs on an arbitrary task. There will be detailed descriptions for each part of the code, similar to a high-quality blog post like this.

Motivation

This will show the possibility of using Hugging Face's PEFT library in a computational biological setting e.g with DNA Language Models to solve a task.

Your contribution

I'll submit a PR if this is a good idea.

Question: Should I place the notebook in the examples folder?

BenjaminBossan commented 3 weeks ago

Thanks for suggesting to provide such a notebook. It would be indeed a good addition for our examples in the examples/ directory.

Note that it looks like people had issues with ESM2 in the past (#1023). If you found a solution, that would be really helpful.