llama-guard eval scripts

meta-llama / PurpleLlama

Set of tools to assess and improve LLM security.

Other

2.73k stars 453 forks source link

llama-guard eval scripts #10

Closed alexbie98 closed 7 months ago

alexbie98 commented 9 months ago

Do you plan to release the eval scripts for the results in the paper and here (https://github.com/facebookresearch/PurpleLlama/blob/main/Llama-Guard/MODEL_CARD.md) for openai moderation and toxicchat?

ujjwalkarn commented 7 months ago

Hi there, Llama recipes provides a script for running Llama Guard inference. We then use sklearn's precision_score, recall_score, f1_score, average_precision_score to compute the metrics. Please let us know if this doesn't help!

dirtycomputer commented 5 months ago

How to get score like this:https://platform.openai.com/docs/guides/moderation/overview