OPTML-Group / Unlearn-Sparse

[NeurIPS23 (Spotlight)] "Model Sparsity Can Simplify Machine Unlearning" by Jinghan Jia*, Jiancheng Liu*, Parikshit Ram, Yuguang Yao, Gaowen Liu, Yang Liu, Pranay Sharma, Sijia Liu
MIT License
62 stars 7 forks source link

cannot find the mask model #3

Closed MrLinNing closed 9 months ago

MrLinNing commented 9 months ago

Hello @ljcc0930 , I'm very happy to read your excellent work. I wonder to ask you which file is the mask model? when i run the retrain after pruning,

python -u main_forget.py --save_dir "./cifar10_results" \
    --mask "./cifar10_results/1model_SA_best.pth.tar" \
    --unlearn retrain --num_indexes_to_replace 4500 --unlearn_epochs 160 \
    --unlearn_lr 0.1 2>&1 | tee -a ./logs/Retrain.log

i have this issue

setup random seed = 2
4500
setup random seed = 2
40500
Pruning with custom mask (all conv layers)
* remain weight ratio =  50.0 %
Traceback (most recent call last):
  File "/home/rram/anaconda3/lib/python3.9/site-packages/torch/serialization.py", line 348, in _check_seekable
    f.seek(f.tell())
AttributeError: 'NoneType' object has no attribute 'seek'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/machine_unlearning/examples/Unlearn-Sparse/main_forget.py", line 248, in <module>
    main()
  File "/machine_unlearning/examples/Unlearn-Sparse/main_forget.py", line 151, in main
    unlearn_method(unlearn_data_loaders, model, criterion, args)
  File "/machine_unlearning/examples/Unlearn-Sparse/unlearn/impl.py", line 63, in _wrapped
    initialization = torch.load(
  File "/anaconda3/lib/python3.9/site-packages/torch/serialization.py", line 771, in load
    with _open_file_like(f, 'rb') as opened_file:
  File "/anaconda3/lib/python3.9/site-packages/torch/serialization.py", line 275, in _open_file_like
    return _open_buffer_reader(name_or_buffer)
  File "/anaconda3/lib/python3.9/site-packages/torch/serialization.py", line 260, in __init__
    _check_seekable(buffer)
  File "/anaconda3/lib/python3.9/site-packages/torch/serialization.py", line 351, in _check_seekable
    raise_err_msg(["seek", "tell"], e)
  File "/anaconda3/lib/python3.9/site-packages/torch/serialization.py", line 344, in raise_err_msg
    raise type(e)(msg)
AttributeError: 'NoneType' object has no attribute 'seek'. You can only torch.load from a file that is seekable. Please pre-load the data into a buffer like io.BytesIO and try to load from it instead.

I have the trained files after run your pruning code image

Can you help me? I'm looking forward to your reply~

jinghanjia commented 9 months ago

Thank you for bringing this to our attention. The mask path is correct, but there may be some issues with using rewind_epoch. By default, the rewind_epoch is set to zero in arg_parser.py, which is the typical setting for most use cases.

However, if the rewind epoch is set to a value other than zero, it is then necessary to provide a corresponding rewind_path as indicated in this section of arg_parser.py. The rewind_path should point to the model checkpoint corresponding to the specified rewind epoch.

To address this, you have two options:

  1. Manually Set the Rewind Epoch to Zero: This is the default and preferred setting if you do not wish to use a specific rewind point. It simplifies the process as there's no need to specify a rewind_path.

  2. Provide a rewind_path: If you choose to use a non-zero rewind epoch, ensure that you also provide a valid rewind_path. This path should lead to the appropriate model checkpoint that corresponds to the specified rewind epoch.

In summary, make sure to align the rewind_epoch setting with the corresponding rewind_path to ensure the framework functions as intended. If you're opting for the default behavior (rewind epoch of zero), no additional action is needed regarding the rewind_path.

If you still have trouble with the unlearning method, please feel free to ask.

Thank you,