merge_and_unload docs do not clarify behaviour for quantized base models

[RonanKMcGovern]

System Info

NA

Who can help?

@BenjaminBossan could you add a note to the docs to explain the default behaviour, and also any work arounds (e.g. loading a base model and dequantizing and loading the adapter and then merging) for best performance? Thanks

https://huggingface.co/docs/peft/main/en/package_reference/lora#peft.LoraModel.merge_and_unload

Information

• [X] The official example scripts
• [ ] My own modified scripts

Tasks

• [X] An officially supported task in the examples folder
• [ ] My own task or dataset (give details below)

Reproduction

NA

Expected behavior

NA

[BenjaminBossan]

I agree that the information is a bit sparse. Could you expand on what exactly you would like to see? What is the workaround for that you mentioned, do you mean quantization methods that don't support merging? What type of performance do you have in mind?