search

OFA-Sys / gsm8k-ScRel

Codes and Data for Scaling Relationship on Learning Mathematical Reasoning with Large Language Models
https://arxiv.org/abs/2308.01825
219 stars 16 forks source link

Is MuggleMath dataset suitable for pre-training? #20

Closed JingyiWang3 closed 10 months ago

JingyiWang3 commented 11 months ago

Hi，thank you for your excellent work! I would like to know if augmented datasets like MuggleMath or RFT are suitable for pre-training?

GanjinZero commented 10 months ago

RFT is used for pre-training as we stated in Qwen.