Request for Help: Implementation of Random Sparsification with ConvNet

[ParthS007]

I am training my ConvNet model with OCT Data and analysing the privacy spent using Opacus by implementing the Random Sparsification and here the implementation uses backPACK for calculating gradients and specifically uses Batchgrad.

I am blocked in testing the paper approach with my model.

Please reproduce using this repo: Github

Steps to reproduce the behavior:

1. python code/train_with_rs_opacus.py

Screenshot

Warning in the logs:

Using a non-full backward hook when the forward contains multiple autograd Nodes is deprecated and will be removed in future versions. This hook will be missing some grad_input. Please use register_full_backward_hook to get the documented behavior.

Environment

• PyTorch Version (e.g., 1.0): 2.3.0
• OS (e.g., Linux): MacOS M2
• How you installed PyTorch (conda, pip, source): pip
• Build command you used (if compiling from source):
• Python version: 3.11.8
• CUDA/cuDNN version:
• GPU models and configuration:
• Any other relevant information: backpack-for-pytorch==1.6.0
• Opacus Version: 1.4.1

Any pointers in the direction to resolve the error would be great! Thanks :)

[JunyiZhu-AI]

Thank you for reaching out!

I apologize for not being able to immediately identify the source of the error you're encountering. It appears to be related to the per-example gradient computation. Please note that the Backpack package supports specific architectures for the version I was using. Given the specifics of your issue, I recommend consulting the Backpack or Opacus teams, as they will likely be able to provide more specialized assistance.