search

wutaiqiang / MoSLoRA

79 stars 8 forks source link

Code path of MoSLoRA #1

Closed vanity1129 closed 2 months ago

vanity1129 commented 2 months ago

Can you provide a specific path to quickly locate the location of MoSLoRA

wutaiqiang commented 2 months ago

https://github.com/wutaiqiang/MoSLoRA/blob/bba79b5491742df44e740e7348a265a0954da903/visual_instruction_tuning/peft/tuners/lora/layer.py#L577

set "self.lora_use_mixer==True" to start moslora

wutaiqiang commented 2 months ago

or you can install this github:

https://github.com/wutaiqiang/peft

to replace the official peft. Just set use_moslora=True in LoRA config and you can start it easily.