search

OFA-Sys / gsm8k-ScRel

Codes and Data for Scaling Relationship on Learning Mathematical Reasoning with Large Language Models
https://arxiv.org/abs/2308.01825
212 stars 16 forks source link

SFT #1

Closed eyuansu62 closed 1 year ago

eyuansu62 commented 1 year ago

Does SFT contain instruction tuning process?

GanjinZero commented 1 year ago

Only gsm8k fine tuning.

eyuansu62 commented 1 year ago

Have you ever planned to verify your findings on more LLM such as codeLLM? Cause I guess codeLLM may points to different conclusion.

GanjinZero commented 1 year ago

Not actually. We will now focus on 65B and 70B LLaMA to verify more scaling related conclusions.