search

PINTO0309 / MobileNet-SSD-RealSense

[High Performance / MAX 30 FPS] RaspberryPi3(RaspberryPi/Raspbian Stretch) or Ubuntu + Multi Neural Compute Stick(NCS/NCS2) + RealSense D435(or USB Camera or PiCamera) + MobileNet-SSD(MobileNetSSD) + Background Multi-transparent(Simple multi-class segmentation) + FaceDetection + MultiGraph + MultiProcessing + MultiClustering
https://qiita.com/PINTO
MIT License
365 stars 127 forks source link

Bad performance #12

Closed treinberger closed 5 years ago

treinberger commented 5 years ago

Great work, thank you very much!

In this gist https://gist.github.com/treinberger/c63cb84979a4b3fb9b13a2d290482f4e I ported your code to get the camera input from the PiCam, but I discovered that the framerate is really low (about 1FPS). Have you got any ideas why this could be the case? I ported your code because I currently don't have a usb webcam at hand. My environment is Raspberry Pi 3+, some wideangle camera and the NCS2.

PINTO0309 commented 5 years ago

@treinberger

I can not say anything certain, but I write what I can think. I do not own "Picamera". Places that often become bottlenecks.

  1. OpenCV version.
  2. if cv2.waitKey(1)&0xFF == ord('q'): ---> There is a bug when OpenCV version is old.
  3. Instead of inference processing, preprocessing and post processing.
  4. cv2.imdecode ---> The cost is very high.
  5. cv2.resize(image,(532,400)) ---> The cost is very high.
  6. Compression and decompression operations are very costly.
  7. Enabling OpenCV "VFPV4" improves performance.
  8. If the shooting resolution is high, the bottleneck of OpenCV becomes remarkable.
treinberger commented 5 years ago

I just updated the sourcecode and I'm using imutils now - and guess what - the performance is somewhere between 8x - 10x times better (about 9fps@640x480 and 10fps@320x240)! Feel free to include the snippet in your repo :-)

PINTO0309 commented 5 years ago

@treinberger

Thank you for giving feedback. :smile: Added to TODO. I am vigorously maintaining other repositories, so it takes many days to merge.

treinberger commented 5 years ago

I forked your repo and made further optimizations: https://github.com/treinberger/MobileNet-SSD-RealSense

Now, RPi 3B+ with 1x NCS2 runs at about 15-16FPS :-)

PINTO0309 commented 5 years ago

Thank you, @treinberger

Very nice! The difference of 5 FPS is very big. If you use the OpenVINO API rather than the OpenCV API, it may work a bit faster. Is there a problem with merging your customization program into my program? Unfortunately, I can not test because I broke PiCamera...

treinberger commented 5 years ago

I just updated the USB cam code as well. The only change besides the performance optimization is the increased camera resolution, but feel free to downgrade it again to 320x240 for better comparison to the other results. My Macbook Cam just didn't work with 320x240.

PINTO0309 commented 5 years ago

I repurchased PiCamera V2. In order to confirm the operation of PiCam, I will fix the program. And I will play for a while. https://www.pyimagesearch.com/2016/01/04/unifying-picamera-and-cv2-videocapture-into-a-single-class-with-opencv/

PINTO0309 commented 5 years ago

@treinberger

Complete. 5b11c89edd3f3fc3422697dac92c2f1fcae6267b

RaspberryPi3 B+, NCS2 x1, PiCamera, 320x240 --> 25 FPS - 27 FPS RaspberryPi3 B+, NCS2 x1, PiCamera, 640x480 --> 15 FPS - 17 FPS RaspberryPi3 B+, NCS2 x1, PiCamera, 1280x720 --> 10 FPS ezgif com-video-to-gif 2

MultiStickSSDwithPiCamera_OpenVINO_NCS2.py