search

huggingface / alignment-handbook

Robust recipes to align language models with human and AI preferences
https://huggingface.co/HuggingFaceH4
Apache License 2.0
4.55k stars 393 forks source link

Missing config params on SFT #31

Closed tcapelle closed 10 months ago

tcapelle commented 10 months ago

Hi, Small PR to add the missing warmup and the total number of steps so the training happens correctly. I am also adding info on the GPU requirements ( 80GB Gpus ). <- this is on the main readme =P

image

The link to the experiment