Closed azayz closed 1 year ago
This PR adds an evaluation script that can be run like the following:
python evaluate.py --load_8bit --base_model 'decapoda-research/llama-7b-hf' --lora_weights 'tloen/alpaca-lora-7b' --eval_file 'user_oriented_instructions.jsonl' --eval_limit 20
This allows to specify an evaluation file that contains instrctions, inputs and outputs to compare your model to and you can specify a limit if you don't want to go through the whole evaluation file
This PR adds an evaluation script that can be run like the following:
python evaluate.py --load_8bit --base_model 'decapoda-research/llama-7b-hf' --lora_weights 'tloen/alpaca-lora-7b' --eval_file 'user_oriented_instructions.jsonl' --eval_limit 20This allows to specify an evaluation file that contains instrctions, inputs and outputs to compare your model to and you can specify a limit if you don't want to go through the whole evaluation file