search

Trusted-AI / adversarial-robustness-toolbox

Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams
https://adversarial-robustness-toolbox.readthedocs.io/en/latest/
MIT License
4.61k stars 1.13k forks source link

Inclusion of Huggingface Support #2225

Open GiulioZizzo opened 11 months ago

GiulioZizzo commented 11 months ago

Huggingface is one of the most popular libraries for carrying out machine learning on large pre-trained foundation models. With growing interest and activity in this area, we can aim to include Huggingface as a framework in ART and thus make evaluating, securing, and developing foundation models a seamless experience in ART.

To do so Huggingface will be included as a new estimator and framework which will support the maximum amount of ART tooling. Hence the aim is to make ART the best available library for foundation model robustness evaluations.

reeshipaul commented 11 months ago

I want to work on this issue!

GiulioZizzo commented 11 months ago

Hi @reeshipaul ! Thank you for your interest in ART and this planned PR. We have a good amount of the code developed for this already, targeting a core set of popular ART tools. I think the most effective way to contribute would be that after ART 1.16 is released, Huggingface functionality can be expanded from this core toolset to as many of the attacks and defences in ART as possible which could be a great place to jump in. @beat-buesser what do you think of this strategy going ahead for ART 1.16 - 1.17 and beyond?

beat-buesser commented 11 months ago

Hi @GiulioZizzo Thank you very much for opening this feature issue!

Hi @reeshipaul Nice to meet you and thank you for your interest in ART! I'm sure we can find a way for you to get started to contribute to ART. Have you been working with HuggingFace before?

reeshipaul commented 11 months ago

I have worked on ML Security before and referred to ART before. I was trying my hand on Hugging Face, pretty sure I can pick this up quickly :)

reeshipaul commented 11 months ago

Hi @beat-buesser can you please help me get started with this issue