search

PINTO0309 / MobileNet-SSD-RealSense

[High Performance / MAX 30 FPS] RaspberryPi3(RaspberryPi/Raspbian Stretch) or Ubuntu + Multi Neural Compute Stick(NCS/NCS2) + RealSense D435(or USB Camera or PiCamera) + MobileNet-SSD(MobileNetSSD) + Background Multi-transparent(Simple multi-class segmentation) + FaceDetection + MultiGraph + MultiProcessing + MultiClustering
https://qiita.com/PINTO
MIT License
364 stars 126 forks source link

Latency of NCS1 vs. NCS2 #14

Open Luxonis-Brandon opened 5 years ago

Luxonis-Brandon commented 5 years ago

So this could be a synchronous/asynchronous observer phenomenon, I'm not sure, and will investigate more. That said, I did want to drop in a comparison here of NCS1 (synchronous) vs. NCS2 (asynchronous):

NCS1 (synchronous): https://photos.app.goo.gl/ejezgpvmMz7U53Qu9 NCS2 (asynchronous): https://photos.app.goo.gl/BExZNpWVaTvLA3wn8

Again, I need to check whether this is just an observational thing of synchronous/asynchronous, but wanted to share these in the meantime in case they give an 'ah-ha' moment. :-)

Thanks again, Brandon

PINTO0309 commented 5 years ago

@Luxonis-Brandon

A little improvement of RealSense latency. Commit: https://github.com/PINTO0309/MobileNet-SSD-RealSense/commit/985af1e22379bc2b48e1782ab97d1916a233be34

$ python3 MultiStickSSDwithRealSense_OpenVINO_NCS2.py -mod 0 -numncs 1

As expected, Image resizing (640, 480) -> (300, 300) is very high load. RaspberryPi3 (or OpenCV) preprocessing load is too high. It seems that NCS2's original performance can not be demonstrated sufficiently. If it can lower the input resolution, it will be fairly high performance. It's a difficult problem...

According to feedback of #10 by nunux13, it is suggested that performance can be raised below.

I use https://github.com/umlaeute/v4l2loopback to feed trafic cam view.

[Note]

Luxonis-Brandon commented 5 years ago

Thanks again @PINTO0309 . I'm trying this out now.

Luxonis-Brandon commented 5 years ago

OK, gave it a try. So I think the latency was improved, but the framerate suffered, down to 2-3FPS from ~10-12FPS. I'm uploading videos now.

And also thanks for the reference. As far as I can tell there seems to be something about how OpenVINO/NCS2 is handing the video feed which is introducing much more latency than on the NCSDK with the NCS1.

I'll probably post on the Intel NCS forum (which I haven't posted on yet) to see what folks there and hopefully some Intel staff themselves think about the extra latency using OpenVINO/NCS2.

Thanks again!