Is there any memory-efficient tensorflow Implementation？

liuzhuang13 / DenseNet

Densely Connected Convolutional Networks, In CVPR 2017 (Best Paper Award).

BSD 3-Clause "New" or "Revised" License

4.69k stars 1.06k forks source link

Is there any memory-efficient tensorflow Implementation？ #32

Open feynman233 opened 6 years ago

feynman233 commented 6 years ago

Hi, I have found the same problem about the GPU memory，and read your report（https://arxiv.org/pdf/1707.06990.pdf）. So, is there any memory-efficient tensorflow Implementation？ Thanks very much!

liuzhuang13 commented 6 years ago

Sorry we don't have any memory-efficient tensorflow implementation, as far as I know.

But I think for practical usage, you don't always need a memory-efficient implementation. Note that memory-efficient version still costs you at least the same amout of time. In most of time, if you have a 4-GPU machine, naive implementation is enough. Thanks!

joeyearsley commented 5 years ago

https://github.com/joeyearsley/efficient_densenet_tensorflow

Made using gradient checkpointing like the gpleiss repo does.