YOLOv4-tiny fast prediction, but slow overall progress

[JeremyKeusters]

I'm having a question on the detector test command. When I run it, it works perfectly fine, but it takes quite some time per image. The prediction itself only takes on average 4.6 ms, but it takes a bit over 300 ms before Darknet processes the next image. Is it possible to speed up this process?

Output when the command is started:

./darknet detector test /darknet/cfg/dataset.data /darknet/cfg/model.cfg /darknet/data/yolov4.weights -thresh 0.25 -dont_show -save_labels < /darknet/data/images.txt
CUDA-version: 11020 (12000), cuDNN: 8.1.1, CUDNN_HALF=1, GPU count: 1
CUDNN_HALF=1
OpenCV version: 4.2.0
Cannot load image data/labels/*.png [I accidentally deleted this folder, will add it again but I guess it has no impact on the performance.]
0 : compute_capability = 600, cudnn_half = 0, GPU: Tesla P100-PCIE-16GB
net.optimized_memory = 0
mini_batch = 1, batch = 4, time_steps = 1, train = 0
layer filters size/strd(dil) input output
0 conv 32 3 x 3/ 2 640 x 640 x 3 -> 320 x 320 x 32 0.177 BF

Model size is 640x640 and I'm using a YOLOv4-tiny.

Small update: while opening this issue I realised that my input images are 0.650MB each, and 2,448x2,048. If I resize to 1,224x1,024, they go down to 0.17MB on average, but it still takes more or less 75ms per image to process. Is this expected behaviour due to the resizing that needs to happen before inference? Or should it go faster? Is there a way to do this process more in batch?

Thanks!

[stephanecharette]

1) You're using a repo that is no longer supported. Instead, see https://github.com/hank-ai/darknet 2) Have you tried DarkHelp? It was written to be easier to batch a large number of images and videos. https://github.com/stephanecharette/DarkHelp

[JeremyKeusters]

Hi Stéphane, thanks for your swift reply! I did not know about hank-ai. I'll take a look into DarkHelp.