search

tlc4418 / llm_optimization

A repo for RLHF training and BoN over LLMs, with support for reward model ensembles.
https://arxiv.org/abs/2310.02743
MIT License
26 stars 1 forks source link

Training Reward Model: AttributeError: 'Namespace' object has no attribute 'residual_dropout_lima'. Did you mean: 'residual_dropout'? #5

Closed RylanSchaeffer closed 2 months ago

RylanSchaeffer commented 2 months ago

I'm trying to train a reward model using the given default configurations but I hit an odd error:

Command:

accelerate launch --config_file configs/accelerate_config.yaml src/reward_modeling/training/trainer_rm.py --configs defaults_rm rm-pythia-44m

Error:

Traceback (most recent call last):
  File "src/reward_modeling/training/trainer_rm.py", line 351, in <module>
    main()
  File "src/reward_modeling/training/trainer_rm.py", line 209, in main
    model = get_model(training_conf, tokenizer)
  File "miniconda3/envs/reward_modeling_env/lib/python3.10/site-packages/model_training/utils/utils.py", line 354, in get_model
    residual_dropout_lima=conf.residual_dropout_lima,
AttributeError: 'Namespace' object has no attribute 'residual_dropout_lima'. Did you mean: 'residual_dropout'?
RylanSchaeffer commented 2 months ago

It might be a bug with Open Assistant: https://github.com/LAION-AI/Open-Assistant/issues/3690

RylanSchaeffer commented 2 months ago

I found I needed to add residual_dropout_lima: false to defaults_rm inside config_rm.yaml

sheikhshafayat commented 1 month ago

Thanks your solution worked for me!