clarification on config std and mean calculation

[karthik-nexusflow]

Hi Team, we would like to drop in our reward model and would like to how the following values were calculated and added to config ? was this calculated by running reward inference on entire train reward dataset ?

if hasattr(config, "mean"):
                self.mean[0] = config.mean
                self.std[0] = config.std

[hijkzzz]

The mean and std are calculated using the testing data of RM (during RM training stage)

[wuxibin89]

https://github.com/OpenLLMAI/OpenRLHF/blob/main/openrlhf/trainer/rm_trainer.py#L219-L226

[karthik-nexusflow]

Great thanks