search

OWASP / www-project-ai-security-and-privacy-guide

OWASP Foundation Web Respository
206 stars 53 forks source link

Have a vulnerable AI project :-)? #6

Open commjoen opened 1 year ago

commjoen commented 1 year ago

Hi there! This is a great project! Thank you for documenting it so nicely! To make it more practical for those that might have trouble understanding some of the sources referenced: will there ever be a “vulnerable AI” project where we can make some of the attacks described more practical :-)?

robvanderveer commented 1 year ago

Robert Sullivan is writing a few articles/tutorials with code to practice attacks. https://www.linkedin.com/pulse/five-security-risks-every-machine-learning-engineer-must-sullivan/

robvanderveer commented 1 year ago

And thanks. Let's discuss this idea on slack or in a call Jeroen.

mspsullivan commented 1 year ago

True. Your interest is well-timed. I see a need for a project that can demonstrate top vulnerabilities, then train defenders to block or detect them. I have two examples (#2 to be published soon) and a third in the works. For now, I suggest we share examples we have seen & had sucess running. I have looked at Adversarial Robustness Toolbox (ART ), there is also Counterfit from Microsoft.

Robert.

On Fri, Mar 24, 2023, 1:51 AM Jeroen Willemsen @.***> wrote:

Hi there! This is a great project! Thank you for documenting it so nicely! To make it more practical: will there ever be a “vulnerable AI” project where we can make some of the attacks described more practical :-)?

— Reply to this email directly, view it on GitHub https://github.com/OWASP/www-project-ai-security-and-privacy-guide/issues/6, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAP2CKL34O7W2NMQCSX3OKDW5UY6DANCNFSM6AAAAAAWGD3T4Y . You are receiving this because you are subscribed to this thread.Message ID: @.***>

mchechik commented 9 months ago

though less well-timed than the OP, WithSecure Labs recently published the damn-vulnerable-llm-agent - a vulnerable chatbot (see project readme for more info)