copy.deepcopy() breaks when pruning is set on sequential

[Ge0rges]

🐛 Bug

Given a a subclass of nn.module which has nn.Sequential() parameters. If we register a pruning method on a subset of those parameters, deep_copy fails.

To Reproduce

Steps to reproduce the behavior:

1. Subclass nn.module
2. Create an attribute and set it to nn.Sequential(ModuleList)
3. Register a pruning method on the parameters within ModuleList

class tester(nn.Module):
    def __init__(self):
        # Encoder
        self.tester = nn.Sequential(*encoder_layers)

        # Set the pruning method
        parameters_to_prune = []
        for param in self.tester:
            if isinstance(param, nn.Linear):
                parameters_to_prune.append((param, "weight"))

        # Add pruning
        prune.global_unstructured(
            parameters_to_prune,
            pruning_method=prune.L1Unstructured,
            amount=pruning_threshold
        )

def main():
    model = tester()
    copy = copy.deepcopy(model)

Below is a stack trace:

File "/home/gio/Documents/KNet/src/main_scripts/den_trainer.py", line 76, in train_tasks model_copy = copy.deepcopy(self.model).to(self.device) if with_den else None File "/usr/lib/python3.7/copy.py", line 180, in deepcopy y = _reconstruct(x, memo, rv) File "/usr/lib/python3.7/copy.py", line 280, in _reconstruct state = deepcopy(state, memo) File "/usr/lib/python3.7/copy.py", line 150, in deepcopy y = copier(x, memo) File "/usr/lib/python3.7/copy.py", line 240, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/usr/lib/python3.7/copy.py", line 180, in deepcopy y = _reconstruct(x, memo, rv) File "/usr/lib/python3.7/copy.py", line 306, in _reconstruct value = deepcopy(value, memo) File "/usr/lib/python3.7/copy.py", line 180, in deepcopy y = _reconstruct(x, memo, rv) File "/usr/lib/python3.7/copy.py", line 280, in _reconstruct state = deepcopy(state, memo) File "/usr/lib/python3.7/copy.py", line 150, in deepcopy y = copier(x, memo) File "/usr/lib/python3.7/copy.py", line 240, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/usr/lib/python3.7/copy.py", line 180, in deepcopy y = _reconstruct(x, memo, rv) File "/usr/lib/python3.7/copy.py", line 306, in _reconstruct value = deepcopy(value, memo) File "/usr/lib/python3.7/copy.py", line 180, in deepcopy y = _reconstruct(x, memo, *rv) File "/usr/lib/python3.7/copy.py", line 280, in _reconstruct state = deepcopy(state, memo) File "/usr/lib/python3.7/copy.py", line 150, in deepcopy y = copier(x, memo) File "/usr/lib/python3.7/copy.py", line 240, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/usr/lib/python3.7/copy.py", line 161, in deepcopy y = copier(memo) File "/home/gio/Documents/KNet/venv/lib/python3.7/site-packages/torch/tensor.py", line 44, in deepcopy raise RuntimeError("Only Tensors created explicitly by the user " RuntimeError: Only Tensors created explicitly by the user (graph leaves) support the deepcopy protocol at the moment

Expected behavior

Does not crash.

Environment

• PyTorch Version: 1.5.0
• OS: Ubuntu 16.10
• How you installed PyTorch: pip
• Python version: 3.7
• CUDA/cuDNN version: None
• GPU models and configuration: None
• Any other relevant information: None

cc @albanD @mruberry @jbschlosser

[albanD]

cc @mickypaganini might be interested in this.

[Ge0rges]

One workaround I can imagine is removing these before the deep_copy and then adding it back right after.

[mickypaganini]

This makes sense... Reparametrizing the network necessarily results in the creation of new parameters in terms of the leaf params. Now, I don't know exactly why in deepcopy in tensor.py we explicitly check for leaves-only, and whether that can be relaxed.

Yes, workaround would be to call prune.remove before the deepcopy (but that might not be suitable if the masks or the original parameters are needed), or to copy first and then prune.

[Ge0rges]

@mickypaganini What do you mean by reparametrizing? Deep_copy shouldn't affect the parameters that currently exist correct? So while the prune function is parameter dependent, I don't see why it couldn't also be carried over.

I have no problem implementing that workaround, but from a software engineering pov it seems to me like "hack" more than anything.

[mickypaganini]

Pruning is a reparametrization. Pruning affects the parameters by rewriting them in terms of other tensors. Deepcopy doesn't affect the parameters that exist. However, it is designed to work only on leaf tensors, and not on derived tensors. That's the first thing it checks, and if that fails, it returns the error you see. I'm sure there's a good reason why pytorch only allows deepcopy of leaf params, but that makes it such that networks reparametrized in this way cannot be copied automatically.

[Ge0rges]

Thank you for that explanation. Given that this has been looked at, and a solution proposed i will close the issue.