search

LukasBommes / mv-extractor

Extract frames and motion vectors from H.264 and MPEG-4 encoded video.
MIT License
296 stars 58 forks source link

working with low frame rates #11

Closed b1q2 closed 2 years ago

b1q2 commented 2 years ago

Hi, I'm trying your project mv-extractor to get timestamp from the camera. While in 30 FPS it works very nice, on 2 FPS I have a delay of about 3-4 seconds between the time that the camera prints on the frame to the timestamp that returns from the mv-extractor. is it possible to get high accuracy in low rates or it is only working on high rates? I added an image from a camera I have: the white text on the image is the time that the camera prints on the frame. the blue text is the timestamp that arrived from the camera. you can see a time diff of 4 seconds thanks.

img

LukasBommes commented 2 years ago

Hey b1q2,

is that an IP camera running on the RTSP protocol? And if so, does it support the ONVIF standard? This is necessary because mv-extractor requires the camera to send RTCP sender reports to calculate the mapping between stream time and wall time.

But I honestly tried this feature only on the camera that I was using in the project back then. So, I don't want to give any guarantees that this works for all cameras. I may also remove this feature in the future as it is not really related to the motion vector extraction. I only included it, because I used the mv-extractor as part of a larger project and had to move the timestamp estimation somewhere.

b1q2 commented 2 years ago

Hi Lukas, Thanks for yor reply. I'm using 'Dahua' as my IP camera. it seems that it is running on RTSP and supports RTCP reports. in the code I'm running debug mode and I get theese two values both true:

  1. packet.synced = 1
  2. this->is_synced = 1

Can you tell us what camera you used getting these good results, and what was the frame-rate? Do you know if the timestamp that returning from the camera is the time of the 'capturing' or the time of the 'sending' of the packet?

thanks.

LukasBommes commented 2 years ago

Hey b1q2,

excuse my slow response... I used the Dahua DH-SD22204T-GN and DH-SD22204T-GN. Regarding the frame rate, I am not sure anymore. Most likely it was configured to run at 1080p resolution with 30 Hz frame rate.

The camera sends both RTSP packets and RTCP sender reports. The RTSP packets contain only the stream time while the RTCP sender reports contain a mapping from the stream time to UTC time. From this, the UTC time at which the RTSP packet was assembled by the camera can be derived. And that is what mv-extractor provides you with in the "timestamp" value. I am honestly not sure, why this approach should break down when the frame rate is slowed.

Whether the timestamp in the RTSP packet represents the time of capture or sending, I am not sure. That should be specified in the RTSP protocol and if not it is probably left to the camera manufacturer to decide. However, the latency within the camera is probably in the order of milliseconds, whereas I encountered network latencies in the order of 30 seconds.

LukasBommes commented 2 years ago

Closing due to inactivity. Feel free to reopen for further discussion.