A slim tensorflow wrapper that provides syntactic sugar for tensor variables. This library will be helpful for practical deep learning researchers not beginners.
MIT License
372
stars
63
forks
source link
Allow passing of parameter for variable initialization #32
Currently those methods are automatically using he_uniform, with assumed scale of 1. This causes problems on large shaped objects, e.g. at some input / outputs I get scale of 0.005 for the uniform method, which causes the network to misbehave and dead neurons to appear (gradients close/equal to 0).
There's no other trivial way to change the initialization methodology except editing the library code.
Allow passing of an opt parameter for variable initialization (scale) in the conv1d, aconv1d, embed, etc. methods (can be found here: https://github.com/buriburisuri/sugartensor/blob/master/sugartensor/sg_layer.py).
Currently those methods are automatically using he_uniform, with assumed scale of 1. This causes problems on large shaped objects, e.g. at some input / outputs I get scale of 0.005 for the uniform method, which causes the network to misbehave and dead neurons to appear (gradients close/equal to 0).
There's no other trivial way to change the initialization methodology except editing the library code.