For the Data_Extraction Attack

QinbinLi / LLM-PBE

A toolkit to assess data privacy in LLMs (under development)

MIT License

37 stars 4 forks source link

For the Data_Extraction Attack #5

Open Zuo-Lihan opened 4 days ago

Zuo-Lihan commented 4 days ago

After run 'python -m attacks.DataExtraction.extract_enron_local --model EleutherAI/pythia-14m --arch EleutherAI/pythia-14m --peft none',

the extraction will obtain a generation_file but I do not see a metric.py to evaluate the generation. May I ask for your help with offering the evaluations of 'attacks.DataExtraction'?

QinbinLi commented 2 days ago

Hi @Zuo-Lihan ,

You can refer to https://github.com/QinbinLi/LLM-PBE/blob/main/notebooks/metric_data_extraction.ipynb. You may change read_all_models function to load your generated files.