search

yaojin17 / Unlearning_LLM

[ACL 2024] Code and data for "Machine Unlearning of Pre-trained Large Language Models"
MIT License
28 stars 0 forks source link

Question about train/val split. #5

Closed tbozhong closed 2 months ago

tbozhong commented 2 months ago

Hi there,

I am currently reproducing your results. However, the dataset used during the train and val phases seems the same. I am curious to understand if this is a common setting or an established convention when working with LLM unlearning.

I'm looking forward to hearing back from you!

yaojin17 commented 2 months ago

Thank you for your valuable question! To the best of my knowledge, current LLM unlearning research usually follows the procedures below. First, researchers identify a set of samples to be forgotten, then perform unlearning algorithms on the forget set and the model. After unlearning, practitioners evaluate the unlearned model on the forget set and usually also measure its performance on a set of retained samples to validate the model's utility. Therefore, there doesn't exist a traditional train and validation set like those in machine learning research. The main difference is that the forget set is used for both unlearning and evaluation, while the retained set is only used for evaluation. If you have further questions, do not hesitate to reach out!

tbozhong commented 2 months ago

Thank you so much for your help! Your solution perfectly solved my problem, and I am very grateful.