search

AlexeyAB / darknet

YOLOv4 / Scaled-YOLOv4 / YOLO - Neural Networks for Object Detection (Windows and Linux version of Darknet )
http://pjreddie.com/darknet/
Other
21.79k stars 7.96k forks source link

How to decrease VPU memory footprint of darknet? #3799

Open LukeAI opened 5 years ago

LukeAI commented 5 years ago

Hey @AlexeyAB I was wondering if you had any tips for decreasing the video memory footprint of darknet without decreasing the network size or changing the cfg? I have a 2080Ti and I'm using libdarknet.so wrapped in python. I note that darknet uses 1867MB video memory or over 2000MB when compiled with CUDNN_HALF

If I am only planning to run inference 1 image at a time with yolov3-spp.cfg - I wonder if there are redundant bits of the source that I can trim that are perhaps occupying a bit of VRAM? I realise that darknet is very tightly coded and that there probably aren't but I thought I'd ask....

AlexeyAB commented 5 years ago

@LukeAI Hi,

No. In the Darknet there is no optimizations of VRAM for inference only.

So you can comment all lines with ... update ... and ... delta ... there https://github.com/AlexeyAB/darknet/blob/4c315ea26b56c2bf20ebc240d94386c6e3cc83db/src/convolutional_layer.c#L474-L540

and these lines: https://github.com/AlexeyAB/darknet/blob/4c315ea26b56c2bf20ebc240d94386c6e3cc83db/src/convolutional_layer.c#L529-L530

https://github.com/AlexeyAB/darknet/blob/4c315ea26b56c2bf20ebc240d94386c6e3cc83db/src/convolutional_layer.c#L538-L539

LukeAI commented 5 years ago

Commenting out 529, 530, 538 and 539 brought down memory usage from 2181MiB to 1853MiB (cudnnn_half) - great tip, thankyou! I got mysterious runtime errors with the update and delta lines from 474 to 540 commented out.

LukeAI commented 5 years ago

Is there anything else I can do other than lines 529, 530, 538 and 539 ? If I can decrease by another 22MiB per instance then I'll be able to run everything that I want simultanaously.

AlexeyAB commented 5 years ago

You can try to find lines with ... update ... and ... delta ... in other layers, in - make_maxpool_layer, make_shortcut_layer, .... and comment them