search

huggingface / peft

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.
https://huggingface.co/docs/peft
Apache License 2.0
16.4k stars 1.61k forks source link

[Feature] Add Quantization Support for VeRA Method #2070

Closed ZiadHelal closed 1 month ago

ZiadHelal commented 1 month ago

Feature request

This pull request aims to introduce support for quantization in the VeRA PEFT method. This enhancement is in response to the issue #1921, which highlighted the need for compatibility between VeRA and quantized models.

Motivation

The motivation behind this feature request is to extend the functionality of the VeRA method to support models that have been quantized using techniques such as bitsandbytes. Currently, users face issues with dimension mismatches and incorrect tensor shapes when applying VeRA to quantized models. By adding support for quantization, we can improve the usability and flexibility of VeRA, making it a more powerful tool for a broader range of use cases.

Your contribution

I am willing to submit a pull request with the implementation of this feature. The PR will include:

Please let me know if there are any additional requirements or considerations for this feature. I look forward to contributing this enhancement to the project and am open to any feedback or suggestions. @BenjaminBossan

BenjaminBossan commented 1 month ago

Thanks a lot @ZiadHelal for offering to work on this feature. Please go ahead and submit a PR. You can also create a draft PR that does not yet cover all your points, and I can give early feedback, which is often better than trying to finish everything in one step.