How to train pretrained Llama 2 adapter model

[huyhuyvu01]

- I have a pretrained adapter model using the following args #CUDA_VISIBLE_DEVICES=0 python ./src/train_bash.py --stage pt --model_name_or_path /hdd-6tb/nhanv/LLM-storage/LLM-models/ELYZA-japanese-Llama-2-7b --do_train --dataset oyo_data --template llama2 --finetuning_type lora --lora_target q_proj,v_proj --output_dir ./runs/afternoon12_9 --overwrite_cache --per_device_train_batch_size 4 --gradient_accumulation_steps 4 --lr_scheduler_type cosine --logging_steps 10 --save_steps 1000 --learning_rate 5e-5 --num_train_epochs 5.0 --plot_loss --fp16

- Now i want to supervised fine-tune on the model i just pretrain. I use the following CUDA_VISIBLE_DEVICES=0 python src/train_bash.py \ --stage sft \ --model_name_or_path /home/ntq/vuhuy/NLP/oyo_llama2/LLaMA-Efficient-Tuning/runs/afternoon12_9 \ --do_train \ --dataset oyo_data_sft \ --template llama2 \ --finetuning_type lora \ --lora_target q_proj,v_proj \ --output_dir ./runs/sft_afternoon18_9 \ --overwrite_cache \ --per_device_train_batch_size 4 \ --gradient_accumulation_steps 4 \ --lr_scheduler_type cosine \ --logging_steps 10 \ --save_steps 1000 \ --learning_rate 5e-5 \ --num_train_epochs 7.0 \ --plot_loss \ --fp16

- But it return error OSError: /home/ntq/vuhuy/NLP/oyo_llama2/LLaMA-Efficient-Tuning/runs/afternoon12_9 does not appear to have a file named config.json. Checkout 'https://huggingface.co//home/ntq/vuhuy/NLP/oyo_llama2/LLaMA-Efficient-Tuning/runs/afternoon12_9/main' for available files.

Does i need to merge the adapter model with the base model in order to do sft training?

[hiyouga]

use export_model.py to merge the LoRA adapters into the base model.

[sahilqure]

@huyhuyvu01 Is your loss converging? I have trained on my custom data. My loss is oscillating between 1.5 to 1.6. What could be the possible reason for that?

[huyhuyvu01]

use export_model.py to merge the LoRA adapters into the base model.

Thanks for the answer, i was able to finetune the model after merging them

[huyhuyvu01]

@huyhuyvu01 Is your loss converging? I have trained on my custom data. My loss is oscillating between 1.5 to 1.6. What could be the possible reason for that?

IDK, but i guess check your data then, i pretrain my LLama2 with 1000 sample and finetune it with 10k question/answer pair. Here my loss for both pretrain and finetune, hope it help u.

[hiyouga]

The loss value is acceptable.

[worstkid92]

use export_model.py to merge the LoRA adapters into the base model.

Sorry to interrpt,but may I ask why is this export_model.py in removed? I can find find it in older releases but not now

[hiyouga]

use llamafactory-cli export