New LLaMA-3 Seq. Classfier Model

[hendrydong]

Hi, congrats to your impactful work again.

We found that the LLaMA-3 model also performs well as the Seq. Classfier. Can you please include our latest LLaMA-3-RM into the reward-bench? We uploaded it at sfairXC/FsfairX-LLaMA3-RM-v0.1.

Thanks in advance!

[natolambert]

Did you test to make sure it runs @hendrydong? Mostly need to figure out which version of Transformers to upgrade to? Will be fast via PR if you know the changes needed.

But yeah, thanks!

[hendrydong]

Yes. I have tested it. The pip list is attached as below. I think it is compatible with LLaMA-2. The only added pkg should be for the tokenizer -- tiktokenand LLaMA-2 is based on sentencepiece.

PyYAML                    6.0.1
ray                       2.11.0
referencing               0.34.0
regex                     2024.4.16
requests                  2.31.0
rewardbench               0.1.0.dev0 
rich                      13.7.1
rpds-py                   0.18.0
ruff                      0.4.0
safetensors               0.4.3
scipy                     1.13.0
semantic-version          2.10.0
sentencepiece             0.2.0
setuptools                68.2.2
shellingham               1.5.4
shortuuid                 1.0.13
shtab                     1.7.1
six                       1.16.0
sniffio                   1.3.1
starlette                 0.37.2
svgwrite                  1.4.3
sympy                     1.12
tabulate                  0.9.0
tiktoken                  0.6.0
tokenizers                0.15.2
tomli                     2.0.1
tomlkit                   0.12.0
toolz                     0.12.1
torch                     2.2.2
tqdm                      4.66.2
transformers              4.38.1
triton                    2.2.0
trl                       0.8.5
typer                     0.12.3
typing_extensions         4.11.0
tyro                      0.8.3
tzdata                    2024.1
urllib3                   2.2.1
uvicorn                   0.29.0
wavedrom                  2.0.3.post3
wcwidth                   0.2.13
websockets                11.0.3
wheel                     0.41.2
xxhash                    3.4.1
yarl                      1.9.4

[natolambert]

@hendrydong will do it tmmw if I don't get to it tonight (you know I'm curious too)

[hendrydong]

Thank you so much! Feel free to tell me if you meet any issue. :)

[natolambert]

@hendrydong I'm curious why y'all use sentiment classifier vs sequence classifier? Is it the same under the hood?

[hendrydong]

I'm curious why y'all use sentiment classifier vs sequence classifier? Is it the same under the hood?

This text classification pipeline can currently be loaded from pipeline() using the following task identifier: "sentiment-analysis" (for classifying sequences according to positive or negative sentiments). The "sentiment-analysis" is equivalent to text-classifier in huggingface.

You can refer to https://huggingface.co/docs/transformers/main_classes/pipelines ("text-classification" (alias "sentiment-analysis" available): will return a TextClassificationPipeline.). I think text-classification is a very early task in HF, so the name and terminologies contains some historical characteristics.