A full pipeline to finetune Vicuna LLM with LoRA and RLHF on consumer hardware. Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the Vicuna architecture. Basically ChatGPT but with Vicuna
while I am doing the second last step
Merge Reward adapter into Model: python merge_peft_adapter.py --model_name ./reward_model_vicuna-7b
I got the following error
Traceback (most recent call last):
File "/home/xuan/anaconda3/envs/vicuna_lora/lib/python3.9/site-packages/peft/tuners/lora.py", line 382, in __getattr__
return super().__getattr__(name) # defer to nn.Module's logic
File "/home/xuan/anaconda3/envs/vicuna_lora/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1614, in __getattr__
raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'LoraModel' object has no attribute '_get_submodules'
how to solve this problem?
never mind, the problem is solved by PEFT的版本，目前从git上安装的是 0.3.0.dev0 版本，在merge_peft_adapter的时候有问题，需要切换到peft==0.2.0 (0.3.0.dev0 没有 _get_submodules()这个函数) as mentioend in the github page.
while I am doing the second last step Merge Reward adapter into Model: python merge_peft_adapter.py --model_name ./reward_model_vicuna-7b
I got the following error
how to solve this problem?
never mind, the problem is solved by PEFT的版本，目前从git上安装的是 0.3.0.dev0 版本，在merge_peft_adapter的时候有问题，需要切换到peft==0.2.0 (0.3.0.dev0 没有 _get_submodules()这个函数) as mentioend in the github page.