shekkizh
commented
7 years ago @NeuroRoboticTech Thanks for the feedback. I'm glad that you find the code useful. As far as I know tensorflow allocates all gpu memory when it starts irrespective of how much it uses. This was the case in older version 0.11 and such - not sure if this was changed. This would explain why you see all memory being used in case of titan X. The model by itself is really big with many variables and at inference each "convolution layer" requires memory in the order of the size of image. One way you can try to fit in a smaller GPU is by setting batch size to 1 or working with smaller images/dividing a larger image into multiple smaller regions and performing inference. Hope this is helpful.
Thanks so much for you wonderful work here. I have been able to modify this code to train some really accurate segmentation models. I am now trying to get one of them running on a Jetson TX1, but I am having some issues. I have tensorflow 1.0.1 installed and running correctly on the TX1, but when I try to run the --visualize setting to just do inference I run out of memory. I went back to my regular desktop that has a Titan X Pascal and did some tests using nvidia-smi to try and see how much memory was being used. It appears that even during the inference it is using over 10 Gb of GPU memory on my system.
Here is the output from sudo watch nvidia-smi while doing inference only: Before running: 745MiB / 12183MiB While Running: 11630MiB / 12183MiB
Do you have any idea why that would be happening? Any ideas on how to reduce this to 1.5 Gb or less for the inference? I can see where it would need that much memory for the training, but I am not sure why it would be doing that on the inference.