search

meta-math / MetaMath

MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models
https://meta-math.github.io
Apache License 2.0
387 stars 35 forks source link

The accuracy of MATH test set trained on 7B is not consistent with paper. #27

Open caihaunqai opened 9 months ago

caihaunqai commented 9 months ago

Hello，Using MetaMATH dataset and codes, I reproduce the experiments on 7B base model. Howerver, The accuracy I get is as follows, 17.14%, which is not consistent with yours 19.8%. Is there anything wrong with your results in the paper or anything wrong with my experiments? Can you help me? Thank you.

start=== 0 , end==== 9223372036854775807 length==== 5000 , acc==== 0.1714

2018211801 commented 5 months ago

Hi, I train 100k subset of Metamath on 7B base model, while get 0 with math test and 0.5 with gsm8k. Is there anything wrong with my experiments?