search

ikhlestov / vision_networks

Repo about neural networks for images handling
MIT License
264 stars 122 forks source link

Bias is used?Other Model can't run! #4

Closed JeffreyLinWeiYou closed 7 years ago

JeffreyLinWeiYou commented 7 years ago

  1. Bias is used? I understood your code ,your only used in 'trainsition_layer_to_classes'. I find PyTorch soucecode that is like used 'conv2d +bias'.

  2. Other Model can't run! Your implementation can run two cases.

DenseNet(k = 12) d=40 DenseNet-BC(k = 12) d=100

Others can't run .Because OOM So,Pytorch version has "Memory Efficient Implementation of DenseNets" implementation detail. https://github.com/liuzhuang13/DenseNet/tree/master/models https://github.com/liuzhuang13/DenseNet/tree/master/models Main ideal is that used "share variable". I understood your code ,your try to use 'out' variable.(reduce?) I think that will build another varable in tensorflow graph.

I want to implement reduce memory.Do you have some ideals? I'm main idea that use share variable.(https://www.tensorflow.org/programmers_guide/variable_scope) But I think that has problem in (tf.concat)?

ikhlestov commented 7 years ago

  1. Regarding bias - I was inherited my code from initial Caffe implementation - https://github.com/liuzhuang13/DenseNetCaffe/blob/master/make_densenet.py#L11 - and there are no bias in conv layers

  2. Regarding OOM. All variables that exist in this network received from get_variable method, so they are already shared. This network should work as it is, and have no any other optimization approaches. But it's up to you to create memory optimized version. You may provide pull request if you want. You can take inspiration from authors torch implementation. They are create customized densely connected layer.

JeffreyLinWeiYou commented 7 years ago

@ikhlestov Thanks a lot!!!