search

OpenVisualCloud / Smart-City-Sample

The smart city reference pipeline shows how to integrate various media building blocks, with analytics powered by the OpenVINO™ Toolkit, for traffic or stadium sensing, analytics and management tasks.
BSD 3-Clause "New" or "Revised" License
191 stars 82 forks source link

Live rtsp stream does not contain bounding boxes and analytics #720

Open divdaisymuffin opened 3 years ago

divdaisymuffin commented 3 years ago

I am using live CCTV coverage in the Smart-city demo, so now I have bounding boxes visible in the recording page, in the form of snippet of 1 min videos. But in the home page, the preview is available, there I am getting direct stream from the camera (live-stream) and not the processed output which contains the analytics and bounding boxes for the objects.

So what I want is to have analytics (bounding boxes and labels) on the preview-template with live stream, or to have processed videos at the home page preview section.

Please help me in this issue.

Thanks

xwu2git commented 3 years ago

If using proceed video as preview, there will be about a minute delay. Is this acceptable? The initial sample has it. If you go back a few commits, before webrtc is introduced, you should see this behavior. Adding analytics on top will be very similar to what recording does.

Current preview utilizes webrtc streaming for real-time response, latency about 200ms. The drawback is that timing isn’t fully aligned with the analytics metadata time stamps. Therefore it will be difficult to display analytics on top of webrtc streamed video. We are still exploring possible workarounds.

divdaisymuffin commented 3 years ago

@xwu2git Thanks for the reply, Actually I am working with face recognition, and I want to have analytics like bounding boxes and face labels on the face with real-time response. so, you are suggesting that, either we can have the recording video (1min delay) in the preview-template at the home page or else with the live stream the analytics is a issue with the current release.

Please help me to understand it further.

xwu2git commented 3 years ago

To overlay bounding boxes on top of the video playback, the two streams must use the same time base. The bounding box time stamps are marked by the analytics pipeline. The time base is when the pipeline makes the rtsp connection. The video segments are recorded in the same pipeline using the same time base. Thus the bounding boxes can overlay on top of the recorded segments. Real-time preview, however, goes through a different pipeline, where the webrtc server pulls streams directly from the camera and then pushes to the browser html5 video box. These two pipelines don’t share the same time base thus there is no good way to align the time stamps to overlay bounding boxes on top.

divdaisymuffin commented 3 years ago

Okay thank you @xwu2git for the detailed explanation of the issue.