Gemma special case handled

for-ai / m-rewardbench

Official Code for M-RᴇᴡᴀʀᴅBᴇɴᴄʜ: Evaluating Reward Models in Multilingual Settings

https://m-rewardbench.github.io/

MIT License

16 stars 2 forks source link

Gemma special case handled #20

Closed ShayekhBinIslam closed 3 months ago

ShayekhBinIslam commented 3 months ago

Gemma does not support system prompts and FastChat template does not work in the current setup. Fixed by concatenating system and user prompts using double-newline and applying tokenizer's chat template.

ljvmiranda921 commented 3 months ago

Thank you for the fix!