search

liuzhuang13 / DenseNet

Densely Connected Convolutional Networks, In CVPR 2017 (Best Paper Award).
BSD 3-Clause "New" or "Revised" License
4.69k stars 1.06k forks source link

Is there any memory efficient DenseNet implementation using Tensorflow #36

Open ybsave opened 6 years ago

ybsave commented 6 years ago

So far, I cannot find any memory efficient implementation DenseNet implementation on Tensorflow. In the Torch codes, there are explicit assignment of shared memory. Would you please provide some hints on how to implement this on Tensorflow? Thank you.

liuzhuang13 commented 6 years ago

Sorry I'm not quite sure how to do this in tensorflow. @taineleau Can you help on this? Thanks!

taineleau-zz commented 6 years ago

Hi, please first check our technical report, which should give you enough knowledge to implement a memory-efficient DenseNet. Basically, if you want to implement the memory-efficient version on a NN framework, this framework should allow you to assign the output of a specific operation manually (i.e., you can manually malloc the memory for the output). PyTorch and Caffe support this operation and MXNet partially support this. However, I am not familiar with Tensorflow so I am not sure how much workload it would be if it does not support the malloc things.

ahundt commented 6 years ago

Note I've got an outstanding feature request for the necessary operations on tensorflow itself at https://github.com/tensorflow/tensorflow/issues/12948.

taineleau-zz commented 6 years ago

@ahundt good job! thanks for helping make a feature request on TF.

joeyearsley commented 5 years ago

https://github.com/joeyearsley/efficient_densenet_tensorflow

Made using gradient checkpointing like the gpleiss repo does.