G-z-w
commented
3 months ago One possible approach is to try the adapter api. See https://huggingface.co/docs/transformers/v4.44.2/en/peft#transformers.integrations.PeftAdapterMixin.get_adapter_state_dict
Open leoozy opened 3 months ago
G-z-w
commented
3 months ago One possible approach is to try the adapter api. See https://huggingface.co/docs/transformers/v4.44.2/en/peft#transformers.integrations.PeftAdapterMixin.get_adapter_state_dict
Motivation
save lora weights only.
Related resources
https://github.com/axolotl-ai-cloud/axolotl
Additional context
No response