OpenBMB / UltraFeedback

A large-scale, fine-grained, diverse preference dataset (and models).
MIT License
297 stars 16 forks source link

Missing harmlessness template or wrong reference #13

Open alvarobartt opened 8 months ago

alvarobartt commented 8 months ago

Description

Hi to whoever is reading this 🤗 I saw that within the code snippets you're referencing a harmlessness template, while there's none on the code (see import below)

https://github.com/OpenBMB/UltraFeedback/blob/bf80fd46a8c6ceecc86e8babb1ae8771f26a3cbb/src/data_annotation/annotate_preference.py#L75

Is that intended? Was there a harmlessness category that was dropped during the process? Is that something to be integrated within the next revision of the paper?

Thanks in advance!