search

huggingface / peft

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.
https://huggingface.co/docs/peft
Apache License 2.0
16.46k stars 1.62k forks source link

FIX: Bug in find_minimal_target_modules #2083

Closed BenjaminBossan closed 1 month ago

BenjaminBossan commented 1 month ago

See #2045 for context.

This bug was reported by Sayak and would occur in find_minimal_target_modules if a required suffix had itself as suffix a string that was already determined to be required, in which case this required suffix would not be added.

The fix consists of prepending a "." to the suffix before checking if it is required or not.

On top of this, the algorithm has been changed to be deterministic. Previously, it was not deterministic because a dictionary that was looped over was built from a set, and sets don't guarantee order. This would result in the loop being in arbitrary order.

As long as the algorithm is 100% correct, the order should not matter. But in case we find bugs like this, the order does matter. We don't want bugs to be flaky, therefore it is best to sort the dict and remove randomness from the function.

HuggingFaceDocBuilderDev commented 1 month ago

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.