search

davidtvs / PyTorch-ENet

PyTorch implementation of ENet
MIT License
389 stars 129 forks source link

Activation selection within the bottlenecks in the network #27

Closed heethesh closed 5 years ago

heethesh commented 5 years ago 
if relu:
    activation = nn.ReLU()
else:
    activation = nn.PReLU()

Does doing this ensure that PReLU weights are unique for each instance of activation within the bottlenecks? While trying to trace this network with torch.jit, it gives errors regarding shared weights by nn.PReLU layers within the submodules. Perhaps this should be implemented with copy.deepcopy for all instances?

To follow the original paper more closely, the number of channels can be specified for each PReLU instance to learn a weight per channel as shown here.

davidtvs commented 5 years ago

honestly, never really used this network much with PReLU activations. How can I replicate your issue?

heethesh commented 5 years ago

Was trying to deploy this model to C++ using this tutorial and realized that the instances of PReLU activations are the same.

davidtvs commented 5 years ago

Fixed the problem causing the error. I still have to update the pre-trained models but in the meantime, the changes are in the param_sharing_fix branch.

Try it out and see if it worts now

heethesh commented 5 years ago

@davidtvs Thanks, the tracing works fine now!

davidtvs commented 5 years ago

For future reference - the fix is now on the master branch