search

TUM-CAMP-NARVIS / nvenc_rtsp

NvPipe with RTSP server/client
Other
6 stars 3 forks source link

Encoder/Sink needs to consume image timestamp #1

Closed ulricheck closed 4 years ago

ulricheck commented 5 years ago

The ServerPipeRTSP class needs to handle timestamp metadata correctly.

currently applications call: m_videoPipe->send_frame(img);

but the call signature should be: m_videoPipe->send_frame(u_int64 ts, const cv::Mat& img);

This is needed so that we can transmit the original camera-frame timestamp and not some internal measure.

kyuq commented 5 years ago

As far as I see a standard rtsp header only reserves 4 bytes for the timestamp. This is now implemented. Are 64 bits must-have?

ulricheck commented 5 years ago

so far the best solution we found would be to rshift the timestamp by 16-bit - something like this: Sender:

Measurement::Timestamp ts = Measurement::Timestamp::now();
uint32_t tss = (ts >>16) && 0x0000FFFF;

Receiver:

Measurement::Timestamp now = Measurement::Timestamp::now();
uint64_t ts = (now & 0xFF000000) | ((tss << 16) & 0x00FFFF00);

that way, we have around microsecond accuracy and can tolerate time-differences of minutes between the hosts (with ntp we should be able to get down to subsecond alignment)

later we can play around with the amount of shift we need in order to maximize synchronization ability of our system.

kyuq commented 5 years ago

nvenc_rtsp is a transmission agent and does not alter the provided timestamp.

Instead, these changes probably should go into the ubitrack git repo, device_comm_videostream.

ulricheck commented 5 years ago

agreed.