Misalignment between config_lora.yaml and the model card

huggingface / alignment-handbook

Robust recipes to align language models with human and AI preferences

https://huggingface.co/HuggingFaceH4

Apache License 2.0

4.28k stars 367 forks source link

Misalignment between config_lora.yaml and the model card #34

Open ChenDRAG opened 8 months ago

ChenDRAG commented 8 months ago

Hi, I noticed that in the model card. It says Adam optimizer is used. However, in the config_lora.yaml file, it uses optim: rmsprop. Could you tell me which one is the actual training configuration?

I don't if there are other hyperparameters I didn't notice. Can you align the scripts with the correct model training configuration, please?