Open crisboarna opened 2 years ago
Could it be due to the h264packetizer
? As the output image seems "smudged", the packetization is not configured fully properly ?
I did have to change compared to the streamer
example the following line:
auto packetizer = make_shared<rtc::H264RtpPacketizer>(rtc::H264RtpPacketizer::Separator::Length, rtpConfig);
to
auto packetizer = make_shared<rtc::H264RtpPacketizer>(rtc::H264RtpPacketizer::Separator::LongStartSequence, rtpConfig);
for it to be able to send.
Would something else need modification ?
By default, tracks are just RTP transceivers, they expect RTP packets as input and output RTP packets. Media handlers may be plugged in, for instance to the RTP packetizer allows to input h264 frames which will be packetized internally. However, the library lacks a depacketizer to do the reverse, so you still get RTP packets as output.
Therefore, you would need to reassemble incoming RTP packets (the most reliable way to do so is using the fact that they have the same timestamp), the removing the RTP headers and glue the payloads together before passing the data to the decoder. You could also pass the incoming RTP string as-is to gstreamer or ffmpeg. Otherwise, the decoder will just get the beginning of the frame and reconstruct the rest by duplicating data, that's what you observe here. For correct performance over non-local networks the implementation would require a proper jitter buffer to reorder packets and pace frames correctly. I'm opening an issue to add such an handler to the library: https://github.com/paullouisageneau/libdatachannel/issues/676
You were right about changing the separator to LongStartSequence
, it's actually the most common separator for NAL units.
Also, you don't need to handle string message from tracks, messages will always be binary. The variant is only to expose the same interface as DataChannel
and WebSocket
, which may transmit string messages.
However, if your use case is actually sending still images, you might want to consider sending compressed images over a data channel, as it will make sure images are properly delivered. Media transport is for real-time video at typically 30+ FPS and does not offer reliability.
Hi, I am trying to send via WebRTC server-to-server some H264 data.
On the sending side I am reading a .264 file that is playable on VLC. I also extract the data via ffmpeg for comparison (Image 1).
On the receiving side I am writing the data to a .264 file from which when I extract via same command the images from the container, the image is distorted significantly. (Image 2)
Size wise, the input data is
6017
bytes while on the receiving side the file is6097
bytes.Here comes my first question, is this expected behaviour for the size to be different?
Secondly, comparing the two with binary editor, I can see there is no similarity between the two bitewise. Is my understanding about this data sending wrong or I am missing out on some steps ?
Why is the image received on other side so significantly distorted ?
The 264 containers I am sending contain single images(project requirement) so it being distored is a big issue.
Image 1 - on sender side Image 2 -on receiver side