search

OWASP / www-project-top-10-for-large-language-model-applications

OWASP Foundation Web Respository
Other
573 stars 141 forks source link

LLM TOP10 - ??? #226

Closed fblgit closed 1 year ago

fblgit commented 1 year ago

Hi,

I am not fully sure what I am reading, but they seems to me very different of any other vulnerabilities within OWASP. These seems more like suggestions of a pentest/report to ChatGPT. In fact, I wasnt able to reproduce any of them in oogabooga mistral/llama models. Training Set Poison... Model theft.. MDoS .. Plugins... ? I don't see any torch statement remediation.. but instead very esoteric "peter & the wolf" stories. I do believe on the need of manifestos, guidelines, etc regarding LLM's safety.. and the content is good to illustrate some good patterns and so.. but from that to call these "vulnerabilities" is a bit dramatic. Definitively these are NOT LLM surface/vectors.. but like bad code, naive usage, ways to exploit bad practices of interfaces to condition the model to output something else.. from my perspective, a LLM vector is that one that i send something in the output and it runs a shellcode or smth like that..

If we imply these arguments as part of OWASP, then .. top#1 OWASP should be weak passwords, ppl uploading data on public s3's, ppl exposing /wipe/sda endpoints or stuff like that. From my perspective, an LLM TOP10 should be focus on LLM's and not on chatgpt.. otherwise you should call this "Top 10 ChatGPT OWASP".. Again, i do believe on the good guidelines provided.. i just dont resonate with how propagandistic AI this seems to me.

GangGreenTemperTatum commented 1 year ago

Hi @fblgit

Thanks for reaching out.

I'd be interested to understand how you tested exploits for vulnerabilities from the current top 10 list against oogabooga mistral/llama models to see what we could be missing in our project and some constructive feedback for us to improve on.

Training Set Poison... Model theft.. MDoS .. Plugins... ? I don't see any torch statement remediation..

Please note, the important factor to our project is "OWASP Top 10 for LLM Applications" (strong emphasis), in which the genesis aims to cover a holistic approach to securing an LLM application and not restricted to the LLM itself. See LLM05 Supply Chain as an example of an attack vector which mentions SBOM/MLBOM (XBOM) and thus would incorporate these types of risks:

I also suggest you checkout huntr as an example for package dependency vulnerabilities as well as other OWASP projects such as #project-mlsec-top-10 which are more orientated specifically at MLSecOps.

I do believe on the need of manifestos, guidelines, etc regarding LLM's safety.. and the content is good to illustrate some good patterns and so.. but from that to call these "vulnerabilities" is a bit dramatic.

I respectfully disagree, "Vulnerabilities are flaws in a computer system that weaken the overall security of the device/system." Thus, a vulnerability against an LLM application can be any (not limited to) of the current top 10.

If we imply these arguments as part of OWASP, then .. top#1 OWASP should be weak passwords, ppl uploading data on public s3's, ppl exposing /wipe/sda endpoints or stuff like that.

This refers to infrastructure, network and cloud security which are important factors when applying layers of security against your attack vector. However, similar to OWASP Top 10 API Security Risks – 2023, this was never our primary focus. As an example, looking for infrastructure against our repo, you can see that our vulnerabilities mention but are not limited to mitigation strategies of this nature.

I'll close this one off, but please feel free to reach out to us at any time. This is an open community and we appreciate everyone's contributions which are always valuable towards our mission statement.