Support converting HF reward models to .nemo

NVIDIA / NeMo-Aligner

Scalable toolkit for efficient model alignment

Apache License 2.0

438 stars 48 forks source link

Open odelalleau opened 4 months ago

odelalleau commented 4 months ago

Is your feature request related to a problem? Please describe.

Converting a HF reward model to .nemo doesn't seem to work right now. See discussion in #109 for details.

Describe the solution you'd like

We should have a script to convert HF reward models to .nemo.

pratikkumar018 commented 4 months ago

Hi Olivier,
Thanks for this. Also we would like to have a backward script which can convert .nemo RM to HF.