search

meta-llama / PurpleLlama

Set of tools to assess and improve LLM security.
Other
2.73k stars 453 forks source link

Why there are two folders of `insecure_code_detector`? #36

Closed fuhengwu2021 closed 5 months ago

fuhengwu2021 commented 5 months ago

I thought they were the same. But after careful comparison, I found there are some difference. Are they just different versions? Which one is better? Do we have plan to merge it? Thanks.

SimonWan commented 5 months ago

Hi @fuhengwu2021 , the two versions are kept temporarily since we still have two benchmarks pointing to the old version. However, we do have a plan to merge them, and for your context:

  1. ICD inside CodeShield is the latest and offers the best quality overall.
  2. ICD inside CybersecurityBenchmarks is the older version, and we plan to deprecate it.

You can also find the changing plan with more details in the README for reference.