search

for-ai / m-rewardbench

Official Code for M-RᴇᴡᴀʀᴅBᴇɴᴄʜ: Evaluating Reward Models in Multilingual Settings
https://m-rewardbench.github.io/
MIT License
15 stars 2 forks source link

Adding languages #55

Open peregilk opened 3 weeks ago

peregilk commented 3 weeks ago

Could you provide instructions on how to add additional languages following your procedures? And estimate how much work this would be.

If new languages was added, are you open to include those in m-rewardbench?

RishabhMaheshwary commented 2 weeks ago

Hello @peregilk! M-Rewardbench relies on the translations of English RewardBench to multiple langauges. To add new languages you can use either the google translate script or nllb translation script to translate RewardBench to your language. M-Rewardbench also has a translation task which includes a subset of MAPLE dataset. The details on how to add such examples are in the paper. Let me know if you have any questions!