Some questions about merging the LoRA back to the base model...
Should the LoRA finetuned on the 4-bit GPTQ model be merged back to the fp16 version of the same model?
When merging the LoRA into the fp16 model, is it recommended to use the PeftModel.merge_and_unload method?
Do you expect the generation speed to increase when using the merged model that is GPTQ'ed after merging, compared to the base GPTQ model with LoRA applied on top of it?
Some questions about merging the LoRA back to the base model...
PeftModel.merge_and_unload
method?