search

chujiezheng / LLM-Safeguard

Official repository for ICML 2024 paper "On Prompt-Driven Safeguarding for Large Language Models"
https://arxiv.org/abs/2401.18018
65 stars 6 forks source link

Missing template for token in evaluate.py #3

Closed wwx0622 closed 6 months ago

wwx0622 commented 7 months ago

在evaluate.py中，函数_get_eval_scores中使用了toker.apply_chat_template来生成input_ids。但是在加载toker的过程中，我没有看到为toker设置chat_template的过程。你能提供一个通用的评分模板吗？我想适应千问系列的模型。

In evaluate.py, the function _get_eval_scores uses toker.apply_chat_template to generate input_ids. But during the process of loading the toker, I did not see the process of setting the chat_template for the toker. Can you provide a universal template for scoring? I want to adapt this project to the Qwen series model.

chujiezheng commented 7 months ago

In evaluate.py, the tokenizer is for LlamaGuard, which has a built-in, complex template (see its tokenizer_config.json on hf). So we do not need further processing for chat_template.