Open adventuree-cyber opened 1 month ago
I used the config at opencompass/configs/datasets/MathBench/mathbench_2024_gen_1dc21d.py and set the use_ppl_single_choice = True. I'm not sure if this is the correct config to test a base model like deepseek-math-7b-base.
I recommend you use mathbench_2024_few_shot_mixed_4a3fd4.py for base model evaluation.
mathbench_2024_few_shot_mixed_4a3fd4.py
I used the config at opencompass/configs/datasets/MathBench/mathbench_2024_gen_1dc21d.py and set the use_ppl_single_choice = True. I'm not sure if this is the correct config to test a base model like deepseek-math-7b-base.