search

NeuroTheoryUMD / NDNT

1 stars 2 forks source link

when to use functionals #3

Open jcbyts opened 3 years ago

jcbyts commented 3 years ago

When saving using torch.save, NDN throws an error about the softplus

PicklingError: Can't pickle <built-in function softplus>: import of module 'torch._C._nn' failed

As far as I can tell, this error results because you can't pickle compiled C code (which is what lives in torch.C)

This happens because we're storing functionals (i.e., F.softplus) in the init, so when save is called, it can't pickle those attributes (because they are compiled C code). So, we need to use nn.Modules when storing them in the init. We can use functionals in the forward, but shouldn't store them as attributes of the model.

jcbyts commented 3 years ago

@dbutts I'll fix this when I update the activation function handling