Is your feature request related to a problem? Please describe.
We have posted a paper with codes [RRHF] (https://github.com/GanjinZero/RRHF) that can achieve human alignment without RLHF. RRHF needs 1-2 models compared with PPO needs 4-5 models which can reduce GPU memory consumption significantly in order to scale to larger LLMs. Is it interesting to add this feature to DeepSpeed or DeepSpeed-chat?
Hi @GanjinZero, we are definitely interested and super welcome to contribute! Do you have any plan and details to share or we can figure it out together.
Is your feature request related to a problem? Please describe. We have posted a paper with codes [RRHF] (https://github.com/GanjinZero/RRHF) that can achieve human alignment without RLHF. RRHF needs 1-2 models compared with PPO needs 4-5 models which can reduce GPU memory consumption significantly in order to scale to larger LLMs. Is it interesting to add this feature to DeepSpeed or DeepSpeed-chat?