search

dan246 / VisionFlow

VisionFlow is a backend application for image recognition and notification systems, built with Flask, PostgreSQL, Redis, and Docker. It supports multi-camera setups, real-time image processing, and provides APIs for user authentication, camera management, and notifications via LINE and email.
MIT License
70 stars 1 forks source link

Delay when Stream from RTSP camera link #4

Open xxtkidxx opened 1 day ago

xxtkidxx commented 1 day ago

Hi @dan246 I have got delay about 1 second when using the RTSP link from a HIK camera I tried both your OpenCV Code and FFMPEG Code on Worker Docker How to reduce delay? image

dan246 commented 1 day ago

Hi @xxtkidxx ,

In the project, the FFmpeg decoding method is set to TCP by default: ffmpeg_cmd = [ 'ffmpeg', '-rtsp_transport', 'tcp', '-loglevel', 'error', '-i', camera_url, '-vf', 'fps=1', '-f', 'rawvideo', '-pix_fmt', 'bgr24', # Using 'bgr24' format 'pipe:' ] TCP is reliable but may introduce latency due to its acknowledgment and retransmission mechanisms. If real-time performance is a higher priority for your use case, switching to UDP as the transport protocol can help reduce the delay.

P.S You might be able to further reduce the delay by implementing the following mechanisms: lowering the FPS, adjusting the FPS buffer size, or using FFmpeg's low-latency mode, among others.

xxtkidxx commented 23 hours ago

Hi @dan246 Thanh you for response, My camera (HIKVision) not support UDP protocol. When I uncomment the lines '-rtsp_transport', 'tcp', error has occurred When I use the config as shown in the photo, the delay is reduced to about 0.7s. Frame recognition delay is about 1s It seems like it's the best result right now image

dan246 commented 22 hours ago

Hi @xxtkidxx,

Thank you for your response and for sharing your findings. I understand your situation and the limitations with your camera. I will continue researching decoding techniques to further reduce latency. If I discover any new optimizations, I’ll make sure to update the project.

Likewise, if you make any progress or find a solution, please feel free to share it as well. Thank you!

xxtkidxx commented 22 hours ago

Hi @dan246 I am very interested in your project and willing to contribute my finding. There is a problem of 300ms delay between stream and recognized stream when I use model Yolo11n.pt I checked that the frame processing time on yolo is only 50ms What is your opinion on this? Is it because of your mainloop function in the Docker recognize object? According to me due:

  1. Delay on self.SLEEP_INTERVAL = 0.1 # Set a reasonable sleep interval
  2. The time for "for loop" depends on the maximum calculation time for each camera. In my opinion this needs to be handled better to reduce latency
  3. API call to cam_ctrl over http. I think you need save camera status to Redis
  4. def fetch_image(self, key): on Class ImageStorage takes over 100ms image