Closed athenawisdoms closed 1 year ago
I've completed my first finetune using finetune.py with params
finetune.py
--lora_out_dir=./my_lora/ \ --llama_q4_config_dir=./Neko-Institute-of-Science_LLaMA-30B-4bit-128g \ --llama_q4_model=./Neko-Institute-of-Science_LLaMA-30B-4bit-128g/llama-30b-4bit-128.safetensors \
How should I modify inference.py to do a quick test on the newly trained lora at ./my_lora/?
inference.py
./my_lora/
I think I need to change model_path and config_path, but which directories should I point them to?
model_path
config_path
You can use this function and set lora_path to your model. load_llama_model_4bit_low_ram_and_offload
Works and thanks!
I've completed my first finetune using
finetune.py
with paramsHow should I modify
inference.py
to do a quick test on the newly trained lora at./my_lora/
?I think I need to change
model_path
andconfig_path
, but which directories should I point them to?