search

huggingface / peft

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.
https://huggingface.co/docs/peft
Apache License 2.0
16.53k stars 1.64k forks source link

[FIX] Invalid `None` check for `loftq_config` attribute in `LoraConfig` #2215

Closed sirluk closed 6 days ago

sirluk commented 1 week ago

Fix for #2170

I improved the handling of loftq attributes in the __init__ method of LoraConfig.

Aside from fixing the issue of the None check I added a warning when init_lora_weights != "loftq" but loftq_config is not specified. In that case I think it also makes sense to reset loftq_config to an empty dict.

I added two testcases for the expected Error/Warning is raised.

fyi: on my local machine a lot of gpu tests are failing but I dont think it is related to the changes as the same tests are failing when using the repo state from the main branch.

HuggingFaceDocBuilderDev commented 1 week ago

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

BenjaminBossan commented 1 week ago

Just a note: For some reason, the MacOS CI tests are not running. Last time this happened, it was (probably) some GH issue that resolved itself after waiting a day. I'll check this again tomorrow.