search

openvinotoolkit / open_model_zoo

Pre-trained Deep Learning models and demos (high quality and extremely fast)
https://docs.openvino.ai/latest/model_zoo.html
Apache License 2.0
4.07k stars 1.36k forks source link

CPU performance on RTSP is very slow #183

Closed Shanmugavadivelu7 closed 4 years ago

Shanmugavadivelu7 commented 5 years ago

Hi, I had tested the performance of driver action recognition demo in real time (rtsp).The performance of the cpu is very slow.

Can anyone suggest me a way to overcome this issue?

Thanks in advance.

kchechil commented 5 years ago

@Shanmugavadivelu7 coudl you please provide more details?

Shanmugavadivelu7 commented 5 years ago

Hi @kchechil

The OpenVINO release is openvino2019.1.144 Hard ware : intel i3 8th gen,4GB RAM OS : Ubuntu 16.04 FPS : less than a frame per second Source : 4MP IP camera I have not tried the benchmark_app Whether I have to try benchmark_app

Thanks in advance

Wovchena commented 5 years ago

Hi @Shanmugavadivelu7 Please, try running simpler use case: provide unmodified demo with video input. What performance results do you get in console?

Shanmugavadivelu7 commented 5 years ago

Hi @Wovchena Thanks for your response The performance results have been attached below Screenshot from 2019-06-26 10-03-53

Could you suggest me how to increase the performance level

Thanks in advance.

vladimir-dudnik commented 5 years ago

@Shanmugavadivelu7 what is frame rate you achieve through your rtsp streaming stack (without any OpenVINO processing) ? it seems like you simple get video frames too slowly. If so, that issue is not related to OpenVINO performance.

snosov1 commented 4 years ago

@Shanmugavadivelu7 please, provide more details and reopen, if it's still relevant.

bus710 commented 4 years ago

Summary

I am afraid to open this ticket again but I am having a similar issue as @Shanmugavadivelu7. The input is HikVIsion's IP camera. Different configurations have been applied but no luck yet.

Detail

Working with a USB cam is totally fine. The interactive face detection demo gave me very fine tuned results regardless of the processing unit (CPU - 30 fps and VPU - 7 fps).

Problems started when I started using an IP camera. The camera has been used is DS-2CD2042WD-I from HikVision.

I suspected that the resolution of the video is the root cause in the beginning so that changed it from 2k to 1280x800 but no luck. The delay is 7-10 seconds. I changed the stream from the main to sub (HikVision cameras come with a main and sub stream for a normal clients such as PC and mobile devices respectively). The sub stream offers 640x360 and 352x240. The demo still shows serious lag like 7-10 seconds.

I tried watching the streams with VLC. Surprisingly, it was totally fine. The delay was less than 2 seconds and it is pretty good as a network camera.

After that I doubt that the receiving part as the root cause. Maybe because of format of the stream? I tried H264 and MJPEG.... The results are same.

What else? some Q&A told me that the VideoCapture API of OpenCV can be the root cause:

I have not deeply checked the demo's code yet but hope my report helps to solve our issue somehow.

Thanks in advance.

Added:

After I submitted this comment, I realized that I cannot re-open it. Please share your thought. If there is no response for a while, I should probably open a new ticket to gain some attention but... I don't want to mess up the list :)

alalek commented 4 years ago

The demo still shows serious lag like 7-10 seconds.

Compare camera settings first (between USB and IP cameras models):

Try to disable any processing (inference) completely.

If this helps, then camera frames queue is full. Frames are just waiting for processing in queue.

To reduce latency, you can try "custom" GStreamer pipeline with reduced frames queue (max-buffers=1) and sync=false in appsink element. Also try to reduce capturing FPS - it should not be larger than processing FPS. This can be estimated through processing of the first captured frame only (no more reads from VideoCapture`).

bus710 commented 4 years ago

@alalek, Thanks for your quick response!

I also did some googling (opencv rtsp delay for example) and found that this is common issue than I thought. There are 2 suggestions to avoid this issue:

  1. Making another thread that refreshes the buffer to be the latest (some people said that handling webcam and RTSP in OpenCV is slightly different. Reading webcam gives us the latest but RTSP gives us the oldest... I am not sure if this is true). However, I don't want to modify the working demo right now because it is Friday :sunglasses:.
  2. Using Gstreamer to discard the interim frames sounds like the plan to go for now. I might try it ASAP and... we will see.

Thanks again,

bus710 commented 4 years ago

So eventually I could safely avoid the lag of VideoCapture when an RTSP camera is being used as the video stream, though I have to start a gst-launch process in advance.

The steps for me are:

  1. Using the latest v4l2loopback kernel module (0.12) - https://github.com/umlaeute/v4l2loopback
  2. Changing the ownership of the device file with chown
  3. Enforce the FPS to be 25 with v4l2loopback-ctl set-fps 25 /dev/video0
  4. Run below command:
$ gst-launch-1.0 \
           ! rtspsrc location=rtsp://admin:camera1234@192.168.1.64:554 latency=300 \
           ! rtph264depay ! avdec_h264 ! video/x-raw ! videoconvert \
           ! v4l2sink device=/dev/video0

Then the face recognition demo can get the video input from the loopback webcam and I could enjoy the maximum performance of it.

Thanks for @alalek,