hiyouga / LLaMA-Factory

Unify Efficient Fine-Tuning of 100+ LLMs
Apache License 2.0
25.52k stars 3.16k forks source link

How to use a fine-tuned model to evaluate on a testset and save the output of the model? #4611

Closed bingkunyao closed 4 days ago

bingkunyao commented 4 days ago

Reminder

System Info

Suppose I already have the fine-tuned model and the base model before fine-tuning, and a JSON file containing some test inputs. Is there a way to automatically obtain the model's output for all the test inputs and save the outputs of the fine-tuned model to a file? Thank you.

Reproduction

Suppose the position of the base model is /home/username/Deepseekcoder-Base and the fine-tuned model is /home/username/Deepseekcoder-sft and the testcase file is testcase.json

Expected behavior

No response

Others

No response

hiyouga commented 4 days ago

see https://github.com/hiyouga/LLaMA-Factory/tree/main/examples#batch-predicting-and-computing-bleu-and-rouge-scores