Awesome Large Language Model Unlearning
This repository tracks the latest research on machine unlearning in large language models (LLMs). The goal is to offer a comprehensive list of papers, datasets, and resources relevant to the topic.
[!NOTE] If you believe your paper on LLM unlearning is not included, or if you find a mistake, typo, or information that is not up to date, please open an issue or submit a pull request, and I will be happy to update the list.
Table of Contents
Papers
Methods
2024
- Practical Unlearning for Large Language Models
- Author(s): Chongyang Gao, Lixu Wang, Chenkai Weng, Xiao Wang, Qi Zhu
- Date: 2024-07
- Venue: -
- Code: -
- Learning to Refuse: Towards Mitigating Privacy Risks in LLMs
- Author(s): Zhenhua Liu, Tong Zhu, Chuanyuan Tan, Wenliang Chen
- Date: 2024-07
- Venue: -
- Code: GitHub
- Composable Interventions for Language Models
- Author(s): Arinbjorn Kolbeinsson, Kyle O'Brien, Tianjin Huang, Shanghua Gao, Shiwei Liu, Jonathan Richard Schwarz, Anurag Vaidya, Faisal Mahmood, Marinka Zitnik, Tianlong Chen, Thomas Hartvigsen
- Date: 2024-07
- Venue: -
- Code: GitHub
- MUSE: Machine Unlearning Six-Way Evaluation for Language Models
- Author(s): Weijia Shi, Jaechan Lee, Yangsibo Huang, Sadhika Malladi, Jieyu Zhao, Ari Holtzman, Daogao Liu, Luke Zettlemoyer, Noah A. Smith, Chiyuan Zhang
- Date: 2024-07
- Venue: -
- Code: -
- If You Don't Understand It, Don't Use It: Eliminating Trojans with Filters Between Layers
- Author(s): Adriano Hernandez
- Date: 2024-07
- Venue: -
- Code: GitHub
- Safe Unlearning: A Surprisingly Effective and Generalizable Solution to Defend Against Jailbreak Attacks
- Author(s): Zhexin Zhang, Junxiao Yang, Pei Ke, Shiyao Cui, Chujie Zheng, Hongning Wang, Minlie Huang
- Date: 2024-07
- Venue: -
- Code: GitHub
- To Forget or Not? Towards Practical Knowledge Unlearning for Large Language Models
- Author(s): Bozhong Tian, Xiaozhuan Liang, Siyuan Cheng, Qingbin Liu, Mengru Wang, Dianbo Sui, Xi Chen, Huajun Chen, Ningyu Zhang
- Date: 2024-07
- Venue: -
- Code: GitHub
- Can Small Language Models Learn, Unlearn, and Retain Noise Patterns?
- Author(s): Nicy Scaria, Silvester John Joseph Kennedy, Deepak Subramani
- Date: 2024-07
- Venue: -
- Code: -
- UnUnlearning: Unlearning is not sufficient for content regulation in advanced generative AI
- Author(s): Ilia Shumailov, Jamie Hayes, Eleni Triantafillou, Guillermo Ortiz-Jimenez, Nicolas Papernot, Matthew Jagielski, Itay Yona, Heidi Howard, Eugene Bagdasaryan
- Date: 2024-07
- Venue: -
- Code: -
- Protecting Privacy Through Approximating Optimal Parameters for Sequence Unlearning in Language Models
- Author(s): Dohyun Lee, Daniel Rim, Minseok Choi, Jaegul Choo
- Date: 2024-06
- Venue: ACL 2024 Findings
- Code: -
- Every Language Counts: Learn and Unlearn in Multilingual LLMs
- Author(s): Taiming Lu, Philipp Koehn
- Date: 2024-06
- Venue: -
- Code: GitHub
- Mitigating Social Biases in Language Models through Unlearning
- Author(s): Omkar Dige, Diljot Singh, Tsz Fung Yau, Qixuan Zhang, Borna Bolandraftar, Xiaodan Zhu, Faiza Khan Khattak
- Date: 2024-06
- Venue: -
- Code: GitHub
- Textual Unlearning Gives a False Sense of Unlearning
- Author(s): Jiacheng Du, Zhibo Wang, Kui Ren
- Date: 2024-06
- Venue: -
- Code: -
- Cross-Lingual Unlearning of Selective Knowledge in Multilingual Language Models
- Author(s): Minseok Choi, Kyunghyun Min, Jaegul Choo
- Date: 2024-06
- Venue: -
- Code: GitHub
- SNAP: Unlearning Selective Knowledge in Large Language Models with Negative Instructions
- Author(s): Minseok Choi, Daniel Rim, Dohyun Lee, Jaegul Choo
- Date: 2024-06
- Venue: -
- Code: GitHub
- Soft Prompting for Unlearning in Large Language Models
- Author(s): Karuna Bhaila, Minh-Hao Van, Xintao Wu
- Date: 2024-06
- Venue: -
- Code: GitHub
- Split, Unlearn, Merge: Leveraging Data Attributes for More Effective Unlearning in LLMs
- Author(s): Swanand Ravindra Kadhe, Farhan Ahmed, Dennis Wei, Nathalie Baracaldo, Inkit Padhi
- Date: 2024-06
- Venue: -
- Code: -
- Intrinsic Evaluation of Unlearning Using Parametric Knowledge Traces
- Author(s): Yihuai Hong, Lei Yu, Shauli Ravfogel, Haiqin Yang, Mor Geva
- Date: 2024-06
- Venue: -
- Code: GitHub
- Avoiding Copyright Infringement via Machine Unlearning
- Author(s): Guangyao Dou, Zheyuan Liu, Qing Lyu, Kaize Ding, Eric Wong
- Date: 2024-06
- Venue: -
- Code: GitHub
- RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models
- Author(s): Zhuoran Jin, Pengfei Cao, Chenhao Wang, Zhitao He, Hongbang Yuan, Jiachun Li, Yubo Chen, Kang Liu, Jun Zhao
- Date: 2024-06
- Venue: -
- Code: GitHub
- REVS: Unlearning Sensitive Information in Language Models via Rank Editing in the Vocabulary Space
- Author(s): Tomer Ashuach, Martin Tutek, Yonatan Belinkov
- Date: 2024-06
- Venue: -
- Code: GitHub
- Unlearning with Control: Assessing Real-world Utility for Large Language Model Unlearning
- Author(s): Qizhou Wang, Bo Han, Puning Yang, Jianing Zhu, Tongliang Liu, Masashi Sugiyama
- Date: 2024-06
- Venue: -
- Code: -
- Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Difference
- Author(s): Jiabao Ji, Yujian Liu, Yang Zhang, Gaowen Liu, Ramana Rao Kompella, Sijia Liu, Shiyu Chang
- Date: 2024-06
- Venue: -
- Code: GitHub
- Large Language Model Unlearning via Embedding-Corrupted Prompts
- Author(s): Chris Yuhao Liu, Yaxuan Wang, Jeffrey Flanigan, Yang Liu
- Date: 2024-06
- Venue: -
- Code: GitHub
- Federated TrustChain: Blockchain-Enhanced LLM Training and Unlearning
- Author(s): Xuhan Zuo, Minghao Wang, Tianqing Zhu, Lefeng Zhang, Dayong Ye, Shui Yu, Wanlei Zhou
- Date: 2024-06
- Venue: -
- Code: -
- Cross-Modal Safety Alignment: Is textual unlearning all you need?
- Author(s): Trishna Chakraborty, Erfan Shayegani, Zikui Cai, Nael Abu-Ghazaleh, M. Salman Asif, Yue Dong, Amit K. Roy-Chowdhury, Chengyu Song
- Date: 2024-06
- Venue: -
- Code: -
- RKLD: Reverse KL-Divergence-based Knowledge Distillation for Unlearning Personal Information in Large Language Models
- Author(s): Bichen Wang, Yuzhe Zi, Yixin Sun, Yanyan Zhao, Bing Qin
- Date: 2024-06
- Venue: -
- Code: -
- Toward Robust Unlearning for LLMs
- Author(s): Rishub Tamirisa, Bhrugu Bharathi, Andy Zhou, Bo Li, Mantas Mazeika
- Date: 2024-05
- Venue: ICLR 2024 SeT-LLM Workshop
- Code: -
- Unlearning Climate Misinformation in Large Language Models
- Author(s): Michael Fore, Simranjit Singh, Chaehong Lee, Amritanshu Pandey, Antonios Anastasopoulos, Dimitrios Stamoulis
- Date: 2024-05
- Venue: -
- Code: -
- Large Scale Knowledge Washing
- Author(s): Yu Wang, Ruihan Wu, Zexue He, Xiusi Chen, Julian McAuley
- Date: 2024-05
- Venue: -
- Code: GitHub
- Single Image Unlearning: Efficient Machine Unlearning in Multimodal Large Language Models
- Author(s): Jiaqi Li, Qianshan Wei, Chuanyi Zhang, Guilin Qi, Miaozeng Du, Yongrui Chen, Sheng Bi
- Date: 2024-05
- Venue: -
- Code: -
- To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language Models
- Author(s): George-Octavian Barbulescu, Peter Triantafillou
- Date: 2024-05
- Venue: ICML 2024
- Code: -
- SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearning
- Author(s): Jinghan Jia, Yihua Zhang, Yimeng Zhang, Jiancheng Liu, Bharat Runwal, James Diffenderfer, Bhavya Kailkhura, Sijia Liu
- Date: 2024-04
- Venue: -
- Code: GitHub
- Machine Unlearning in Large Language Models
- Author(s): Kongyang Chen, Zixin Wang, Bing Mi, Waixi Liu, Shaowei Wang, Xiaojun Ren, Jiaxing Shen
- Date: 2024-04
- Venue: -
- Code: -
- Offset Unlearning for Large Language Models
- Author(s): James Y. Huang, Wenxuan Zhou, Fei Wang, Fred Morstatter, Sheng Zhang, Hoifung Poon, Muhao Chen
- Date: 2024-04
- Venue: -
- Code: GitHub
- Eraser: Jailbreaking Defense in Large Language Models via Unlearning Harmful Knowledge
- Author(s): Weikai Lu, Ziqian Zeng, Jianwei Wang, Zhengdong Lu, Zelin Chen, Huiping Zhuang, Cen Chen
- Date: 2024-04
- Venue: -
- Code: -
- Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning
- Author(s): Ruiqi Zhang, Licong Lin, Yu Bai, Song Mei
- Date: 2024-04
- Venue: -
- Code: GitHub
- Localizing Paragraph Memorization in Language Models
- Author(s): Niklas Stoehr, Mitchell Gordon, Chiyuan Zhang, Owen Lewis
- Date: 2024-03
- Venue: -
- Code: -
- The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning
- Author(s): Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D. Li, Ann-Kathrin Dombrowski, Shashwat Goel, Long Phan, Gabriel Mukobi, Nathan Helm-Burger, Rassin Lababidi, Lennart Justen, Andrew B. Liu, Michael Chen, Isabelle Barrass, Oliver Zhang, Xiaoyuan Zhu, Rishub Tamirisa, Bhrugu Bharathi, Adam Khoja, Zhenqi Zhao, Ariel Herbert-Voss, Cort B. Breuer, Samuel Marks, Oam Patel, Andy Zou, Mantas Mazeika, Zifan Wang, Palash Oswal, Weiran Lin, Adam A. Hunt, Justin Tienken-Harder, Kevin Y. Shih, Kemper Talley, John Guan, Russell Kaplan, Ian Steneker, David Campbell, Brad Jokubaitis, Alex Levinson, Jean Wang, William Qian, Kallol Krishna Karmakar, Steven Basart, Stephen Fitz, Mindy Levine, Ponnurangam Kumaraguru, Uday Tupakula, Vijay Varadharajan, Ruoyu Wang, Yan Shoshitaishvili, Jimmy Ba, Kevin M. Esvelt, Alexandr Wang, Dan Hendrycks
- Date: 2024-03
- Venue: -
- Code: GitHub
- Dissecting Language Models: Machine Unlearning via Selective Pruning
- Author(s): Nicholas Pochinkov, Nandi Schoots
- Date: 2024-03
- Venue: -
- Code: -
- Second-Order Information Matters: Revisiting Machine Unlearning for Large Language Models
- Author(s): Kang Gu, Md Rafi Ur Rashid, Najrin Sultana, Shagufta Mehnaz
- Date: 2024-03
- Venue: -
- Code: -
- Ethos: Rectifying Language Models in Orthogonal Parameter Space
- Author(s): Lei Gao, Yue Niu, Tingting Tang, Salman Avestimehr, Murali Annavaram
- Date: 2024-03
- Venue: -
- Code: -
- Towards Efficient and Effective Unlearning of Large Language Models for Recommendation
- Author(s): Hangyu Wang, Jianghao Lin, Bo Chen, Yang Yang, Ruiming Tang, Weinan Zhang, Yong Yu
- Date: 2024-03
- Venue: -
- Code: GitHub
- Guardrail Baselines for Unlearning in LLMs
- Author(s): Pratiksha Thaker, Yash Maurya, Virginia Smith
- Date: 2024-03
- Venue: ICLR 2024 SeT-LLM Workshop
- Code: -
- Deciphering the Impact of Pretraining Data on Large Language Models through Machine Unlearning
- Author(s): Deciphering the Impact of Pretraining Data on Large Language Models through Machine Unlearning
- Date: 2024-02
- Venue: -
- Code: -
- Unmemorization in Large Language Models via Self-Distillation and Deliberate Imagination
- Author(s): Yijiang River Dong, Hongzhou Lin, Mikhail Belkin, Ramon Huerta, Ivan Vulić
- Date: 2024-02
- Venue: -
- Code: GitHub
- Towards Safer Large Language Models through Machine Unlearning
- Author(s): Zheyuan Liu, Guangyao Dou, Zhaoxuan Tan, Yijun Tian, Meng Jiang
- Date: 2024-02
- Venue: -
- Code: GitHub
- Selective Forgetting: Advancing Machine Unlearning Techniques and Evaluation in Language Models
- Author(s): Lingzhi Wang, Xingshan Zeng, Jinsong Guo, Kam-Fai Wong, Georg Gottlob
- Date: 2024-02
- Venue: -
- Code: -
- Unlearnable Algorithms for In-context Learning
- Author(s): Andrei Muresanu, Anvith Thudi, Michael R. Zhang, Nicolas Papernot
- Date: 2024-02
- Venue: -
- Code: -
- Machine Unlearning of Pre-trained Large Language Models
- Author(s): Jin Yao, Eli Chien, Minxin Du, Xinyao Niu, Tianhao Wang, Zezhou Cheng, Xiang Yue
- Date: 2024-02
- Venue: ACL 2024
- Code: GitHub
- Visual In-Context Learning for Large Vision-Language Models
- Author(s): Yucheng Zhou, Xiang Li, Qianning Wang, Jianbing Shen
- Date: 2024-02
- Venue: -
- Code: -
- EFUF: Efficient Fine-grained Unlearning Framework for Mitigating Hallucinations in Multimodal Large Language Models
- Author(s): Shangyu Xing, Fei Zhao, Zhen Wu, Tuo An, Weihao Chen, Chunhui Li, Jianbing Zhang, Xinyu Dai
- Date: 2024-02
- Venue: -
- Code: -
- Unlearning Reveals the Influential Training Data of Language Models
- Author(s): Masaru Isonuma, Ivan Titov
- Date: 2024-01
- Venue: -
- Code: -
- TOFU: A Task of Fictitious Unlearning for LLMs
- Author(s): Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary C. Lipton, J. Zico Kolter
- Date: 2024-01
- Venue: -
- Code: GitHub
2023
- FairSISA: Ensemble Post-Processing to Improve Fairness of Unlearning in LLMs
- Author(s): Swanand Ravindra Kadhe, Anisa Halimi, Ambrish Rawat, Nathalie Baracaldo
- Date: 2023-12
- Venue: NeurIPS 2023 SoLaR Workshop
- Code: -
- Making Harmful Behaviors Unlearnable for Large Language Models
- Author(s): Xin Zhou, Yi Lu, Ruotian Ma, Tao Gui, Qi Zhang, Xuanjing Huang
- Date: 2023-11
- Venue: -
- Code: -
- Forgetting before Learning: Utilizing Parametric Arithmetic for Knowledge Updating in Large Language Models
- Author(s): Shiwen Ni, Dingwei Chen, Chengming Li, Xiping Hu, Ruifeng Xu, Min Yang
- Date: 2023-11
- Venue: -
- Code: -
- Who's Harry Potter? Approximate Unlearning in LLMs
- Author(s): Ronen Eldan, Mark Russinovich
- Date: 2023-10
- Venue: -
- Code: -
- DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Models
- Author(s): Xinwei Wu, Junzhuo Li, Minghui Xu, Weilong Dong, Shuangzhi Wu, Chao Bian, Deyi Xiong
- Date: 2023-10
- Venue: EMNLP 2023
- Code: GitHub
- Unlearn What You Want to Forget: Efficient Unlearning for LLMs
- Author(s): Jiaao Chen, Diyi Yang
- Date: 2023-10
- Venue: EMNLP 2023
- Code: GitHub
- In-Context Unlearning: Language Models as Few Shot Unlearners
- Author(s): Martin Pawelczyk, Seth Neel, Himabindu Lakkaraju
- Date: 2023-10
- Venue: -
- Code: -
- Large Language Model Unlearning
- Author(s): Yuanshun Yao, Xiaojun Xu, Yang Liu
- Date: 2023-10
- Venue: NeurIPS 2023 SoLaR Workshop
- Code: GitHub
- Forgetting Private Textual Sequences in Language Models via Leave-One-Out Ensemble
- Author(s): Zhe Liu, Ozlem Kalinli
- Date: 2023-09
- Venue: -
- Code: -
- Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attacks
- Author(s): Vaidehi Patil, Peter Hase, Mohit Bansal
- Date: 2023-09
- Venue: -
- Code: GitHub
- Separate the Wheat from the Chaff: Model Deficiency Unlearning via Parameter-Efficient Module Operation
- Author(s): Xinshuo Hu, Dongfang Li, Baotian Hu, Zihao Zheng, Zhenyu Liu, Min Zhang
- Date: 2023-08
- Venue: AAAI 2024
- Code: GitHub
- Unlearning Bias in Language Models by Partitioning Gradients
- Author(s): Charles Yu, Sullam Jeoung, Anish Kasi, Pengfei Yu, Heng Ji
- Date: 2023-07
- Venue: ACL (Findings) 2023
- Code: GitHub
- Make Text Unlearnable: Exploiting Effective Patterns to Protect Personal Data
- Author(s): Xinzhe Li, Ming Liu, Shang Gao
- Date: 2023-07
- Venue: -
- Code: -
- What can we learn from Data Leakage and Unlearning for Law?
- Author(s): Jaydeep Borkar
- Date: 2023-07
- Venue: -
- Code: -
- LEACE: Perfect linear concept erasure in closed form
- Author(s): Nora Belrose, David Schneider-Joseph, Shauli Ravfogel, Ryan Cotterell, Edward Raff, Stella Biderman
- Date: 2023-06
- Venue: NeurIPS 2023
- Code: GitHub
- Composing Parameter-Efficient Modules with Arithmetic Operations
- Author(s): Jinghan Zhang, Shiqi Chen, Junteng Liu, Junxian He
- Date: 2023-06
- Venue: NeurIPS 2023
- Code: GitHub
- KGA: A General Machine Unlearning Framework Based on Knowledge Gap Alignment
- Author(s): Lingzhi Wang, Tong Chen, Wei Yuan, Xingshan Zeng, Kam-Fai Wong, Hongzhi Yin
- Date: 2023-05
- Venue: -
- Code: GitHub
2022
- Editing Models with Task Arithmetic
- Author(s): Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, Ali Farhadi
- Date: 2022-12
- Venue: ICLR 2023
- Code: GitHub
- Privacy Adhering Machine Un-learning in NLP
- Author(s): Vinayshekhar Bannihatti Kumar, Rashmi Gangadharaiah, Dan Roth
- Date: 2022-12
- Venue: -
- Code: -
- The CRINGE Loss: Learning what language not to model
- Author(s): Leonard Adolphs, Tianyu Gao, Jing Xu, Kurt Shuster, Sainbayar Sukhbaatar, Jason Weston
- Date: 2022-11
- Venue: -
- Code: -
- Knowledge Unlearning for Mitigating Privacy Risks in Language Models
- Author(s): Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, Minjoon Seo
- Date: 2022-10
- Venue: -
- Code: GitHub
- Quark: Controllable Text Generation with Reinforced Unlearning
- Author(s): Ximing Lu, Sean Welleck, Jack Hessel, Liwei Jiang, Lianhui Qin, Peter West, Prithviraj Ammanabrolu, Yejin Choi
- Date: 2022-05
- Venue: NeurIPS 2022
- Code: GitHub
2021
- DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts
- Author(s): Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, Yejin Choi
- Date: 2021-05
- Venue: ACL 2021
- Code: GitHub
Surveys and Position Papers
2024
- Digital Forgetting in Large Language Models: A Survey of Unlearning Methods
- Author(s): Alberto Blanco-Justicia, Najeeb Jebreel, Benet Manzanares, David Sánchez, Josep Domingo-Ferrer, Guillem Collell, Kuan Eeik Tan
- Date: 2024-04
- Venue: -
- Machine Unlearning for Traditional Models and Large Language Models: A Short Survey
- Author(s): Yi Xu
- Date: 2024-04
- Venue: -
- The Frontier of Data Erasure: Machine Unlearning for Large Language Models
- Author(s): Youyang Qu, Ming Ding, Nan Sun, Kanchana Thilakarathna, Tianqing Zhu, Dusit Niyato
- Date: 2024-03
- Venue: -
- Rethinking Machine Unlearning for Large Language Models
- Author(s): Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Xiaojun Xu, Yuguang Yao, Hang Li, Kush R. Varshney, Mohit Bansal, Sanmi Koyejo, Yang Liu
- Date: 2024-02
- Venue: -
- Eight Methods to Evaluate Robust Unlearning in LLMs
- Author(s): Aengus Lynch, Phillip Guo, Aidan Ewart, Stephen Casper, Dylan Hadfield-Menell
- Date: 2024-02
- Venue: -
2023
- Knowledge Unlearning for LLMs: Tasks, Methods, and Challenges
- Author(s): Nianwen Si, Hao Zhang, Heyu Chang, Wenlin Zhang, Dan Qu, Weiqiang Zhang
- Date: 2023-11
- Venue: -
- Right to be Forgotten in the Era of Large Language Models: Implications, Challenges, and Solutions
- Author(s): Dawen Zhang, Pamela Finckenberg-Broman, Thong Hoang, Shidong Pan, Zhenchang Xing, Mark Staples, Xiwei Xu
- Date: 2023-07
- Venue: -
Blog Posts
- Machine Unlearning in 2024
- Author(s): Ken Liu
- Date: 2024-05
- Deep Forgetting & Unlearning for Safely-Scoped LLMs
- Author(s): Stephen Casper
- Date: 2023-12
Datasets
- RWKU
- Description: This dataset is designed to test large language models' ability to unlearn real-world knowledge. It contains 200 unlearning targets and 13,131 forget probes, including fill-in-the-blank, question-answer, and adversarial attack probes. The task is to forget famous people from Wikipedia. The evaluation suite assesses knowledge memorization and manipulation through regular and adversarial settings, as well as membership inference attacks. It also evaluates general ability, reasoning, truthfulness, factuality, and fluency. The unlearning task does not assume access to the forget or retain corpora, as the pre-training data of most language models is not easily accessible.
- Links: arXiv, Hugging Face
- TOFU
- Description: A synthetic QA dataset of fictitious authors generated by GPT-4. The datasets comes with three splits of the retain/forget sets, including 99/1, 95/5, and 90/10 (in percentage). The dataset also includes questions about real authors and world facts to evaluate the loss of general knowledge after unlearning.
- Links: arXiv, Hugging Face
- WMDP
- Description: A benchmark for assessing hazardous knowledge in biology, chemistry, and cybersecurity, containing about 4000 multiple-choice questions with similar style to MMLU. It also comes with corpora in the three domains.
- Links: arXiv, Hugging Face
- MMLU Subsets
- Description: A task proposed along with the WMDP dataset. The goal is to unlearn (retain) three categories in the MMLU dataset: economics (econometrics and others), physics (math and others), and law (jurisprudence and others). The task requires high-precision unlearning, because the retain sets are categories closely related to the unlearning categories.
- Links: arXiv, Hugging Face
- arXiv, GitHub, and copyrighted books corpus
- Description: A dataset for evaluating approximate unlearning algorithms for pre-trained LLMs. The dataset contains both forget and retain splits of each category, and comes with both in-distribution and general retain sets. The dataset is deisgned for unlearning directly on pre-trained models, as they are random samples from the pre-training dataset of Yi.
- Links: arXiv, Hugging Face