search

allenai / reward-bench

RewardBench: the first evaluation tool for reward models.
https://huggingface.co/spaces/allenai/reward-bench
Apache License 2.0
440 stars 52 forks source link

Add Cloud RMs #173

Open natolambert opened 2 months ago

natolambert commented 2 months ago

See @zankner's repo https://github.com/zankner/CLoud, RM's that think out loud!

scottsuk0306 commented 1 hour ago

Hi @natolambert, I recently trained a generative RM (prometheus-eval/prometheus-RM-Llama-8B-v1.0) based on the CLoud code base and l think the inference using the huggingface transformers can be easily integrated to existing code of reward-bench. Can I try working on this?