search

InternLM / xtuner

An efficient, flexible and full-featured toolkit for fine-tuning LLM (InternLM2, Llama3, Phi3, Qwen, Mistral, ...)
https://xtuner.readthedocs.io/zh-cn/latest/
Apache License 2.0
3.72k stars 302 forks source link

How to perform validation during the fine-tune training process on llava_llama3_8b_instruct_full_clip_vit_large_p14_336_lora_e1_gpu8_finetune? #749

Open J0eky opened 3 months ago

J0eky commented 3 months ago

I have been fine-tuning the llava-llama3-8b-v1_1 model on my own dataset using the llava_llama3_8b_instruct_full_clip_vit_large_p14_336_lora_e1_gpu8_finetune_copy.py script. While the training phase performed well, I observed an absence of any validation process throughout the training, potentially culminating in overfitting concerns. It would be grateful if someone could offer some advice.

hhaAndroid commented 3 months ago

@J0eky The current main branch code does not support validation during training, but you can use the EvaluateChatHook to see the dialogue effect. We currently have the functionality to perform validation during training on other branches, and we will consider merging this in the future.

DwanZhang-AI commented 3 months ago

Same problem, requires the validation code while fine-tuning