search

IS2Lab / S-Eval

S-Eval: Automatic and Adaptive Test Generation for Benchmarking Safety Evaluation of Large Language Models
Other
22 stars 3 forks source link

How to do the automatic safety evaluations #3

Open TianuaXu opened 2 weeks ago

TianuaXu commented 2 weeks ago

Hello!

This is very nice work!

I kindly want to know how to do the automatic safety evaluations. According to the paper, you use a safety critique llm for the evaluations. Will you release the safety critique llm in the future? Or are there any other methods for the automatic safety evaluations?

IS2Lab commented 1 week ago

Thanks for your recognition of our work. At present, we have no plans to release our safety critique llm in the short term. However, we are happy to provide support to help you with safety evaluations. If you require automatic safety evaluations, please provide a jsonl file including traceid, prompt, and the corresponding response and send it to xiaohanyuan@zju.edu.cn (if you have any questions about the required submission files, please contact this email first.) We will evaluate and return the results according to the order of requests.

In addition, for other automatic safety evaluations, we introduced and analyzed various methods in our paper. You can refer to the paper.

Thanks again for your understanding and support!