search

philipperemy / yolo-9000

YOLO9000: Better, Faster, Stronger - Real-Time Object Detection. 9000 classes!
Apache License 2.0
1.18k stars 309 forks source link

FPS very poor #31

Closed foocp closed 5 years ago

foocp commented 5 years ago

Hi, I'm on a 1080Ti and only get between 2.5 and 3fps. Here you asked for the results ofwatch -n0.01 nvidia-dmi. There you go. The Volatile GPU jumps between 0% and 25%, regardless if I measure once every 0,1s or every 3s.

foocp commented 5 years ago
screen shot 2018-09-06 at 17 03 55

Grateful for any tips!

philipperemy commented 5 years ago

@foocp

Not sure why.

With this video: https://textinmotion.tv/help/samples (720p)

https://d2qguwbxlx1sbt.cloudfront.net/TextInMotion-VideoSample-720p.mp4

./darknet detector demo cfg/combine9k.data cfg/yolo9000.cfg ../yolo9000-weights/yolo9000.weights  -prefix output TextInMotion-VideoSample-720p.mp4 -thresh 0.15

I could process 340 frames in 10 seconds. So roughly 34 FPS with my GTX 1070.

I freshly git cloned the repository and followed all the steps.

Hope it helps.

My GPU utilization was constant around ~35-37%.

Every 1.0s: nvidia-smi                                                                                                                         Tue Sep 18 23:07:44 2018

Tue Sep 18 23:07:44 2018
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 396.26                 Driver Version: 396.26                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 1070    Off  | 00000000:01:00.0  On |                  N/A |
|  0%   58C    P2    58W / 200W |   1738MiB /  8116MiB |     36%      Default |
+-------------------------------+----------------------+----------------------+
|   1  GeForce GTX 1070    Off  | 00000000:03:00.0 Off |                  N/A |
|  0%   50C    P8    14W / 200W |     12MiB /  8119MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0      1041      G   /usr/lib/xorg/Xorg                           139MiB |
|    0      1679      G   compiz                                       214MiB |
|    0      1900      G   /opt/teamviewer/tv_bin/TeamViewer             20MiB |
|    0     17707      C   ./darknet                                   1259MiB |
|    0     24106      C   python                                        93MiB |
+-----------------------------------------------------------------------------+
philipperemy commented 5 years ago

@foocp just for info, I see FPS 0.0 (seems to be a bug in darknet) but the images get generated anyway.

philipperemy commented 5 years ago

So can you check with ll *.png | wc -l at every minute and compute the FPS yourself?

philipperemy commented 5 years ago

I'll close it for now since there's no problem on my end

foocp commented 5 years ago

@philipperem sorry, seems my notification got lost somewhere. I should clarify - as in the referenced issue, the problem with the slow fps is when running the demo with a webcam connected via USB.

philipperemy commented 5 years ago

@foocp in that case I guess it has more to do with the framerate of the webcam or the open cv framework itself.

foocp commented 5 years ago

OK, I'll check that out, thanks!

foocp commented 5 years ago

Hi @philipperem, turns out it's also very slow when processing that test video. Here's the output of watch -n0.01 nvidia-smi

The Volatile GPU changes between 10% and 22%.

screenshot from 2018-09-28 12-45-16

Darknet seems to use a version of OpenCV that has no Cuda support. I checked in Python using

>>> import cv2
>>> print(cv2.getBuildInformation())

That would be why it's slow on my machine, right? Now how to get rid of multiple OpenCV installations...

philipperemy commented 5 years ago

Yes it's possible! And in your case I have no ideas on how to get ride of multiple OpenCV. A fresh re-install might be the quickest

foocp commented 5 years ago

I de- and reinstalled all of OpenCV, to no avail. If there's only the self-compiled OpenCV 3.4 (with CUDA support), darknet won't run, as it's missing something. So I guess it wants OpenCV 2, which even PJ Reddie himself says he simply installs from the repository. But of course that won't have special CUDA support.

Yolo v3 runs at least three times faster than Yolo9000 (which is what I need for my project). Am I missing something?

By the way, thanks for your thoughts!

And btw, as a warning to others, don't bother installing OpenCV 3.4.1, there's a known bug that stops it from working with c libraries. Cost me hours...

foocp commented 5 years ago

Update, the problem is that Yolo9000 doesn't run as fast with the latest versions of darknet. I've tested it until the version from April 11, 2018, and I still get 30fps on my 1080Ti. Will try and find out where exactly it stops working well.

philipperemy commented 5 years ago

@foocp thanks for this update. That's interesting. This repo is not linked to the latest version of darknet btw. Would be curious to know why if you found the answer.

foocp commented 5 years ago

oops, turns out the PJ Reddie's darknet and AlexeyAB's fork are a lot more different than I naively assumed. It's the latter that runs very slowly with Yolo9000. The original darknet, including the lastest as of now, works fine. Thanks anyway for your input!

philipperemy commented 5 years ago

Ok happy to know that. That's interesting.