Videochat codec

[Remigius1974]

Hi,

I try to implement a simple video chat with the datachannel data transfer. I looked at the media example and with the gstreamer it works fine. As HTML video component does not support the H264 codec I switched to the VP8 codec. But even with this, the video will not render and the browser (Chrome) complains with this error:

Failed to execute 'appendBuffer' on 'SourceBuffer': This SourceBuffer has been removed from the parent media source

As I understand (and asked google :)) it means, the data does not match the codec and is not recognised.

The code to fill the data on the frontend:

            var mimeCodec = 'video/webm; codecs="vp8"';
            if (MediaSource.isTypeSupported(mimeCodec)) {
                var ms = new MediaSource();
                remoteVideo.src = window.URL.createObjectURL(ms);

                ms.addEventListener('sourceopen', function(e) {
                    var sourceBuffer = ms.addSourceBuffer(mimeCodec);

                    event.channel.addEventListener("message", (msg) => {
                        sourceBuffer.appendBuffer(msg.data);
                    });
                }, false);
            } else {
                console.error("Unsupported media format");
            }

The data is available, I checked it and it is a arrayBuffer.

The code on the server side:

let video = new nodeDataChannel.Video('video', 'RecvOnly');
video.addVP8Codec(97);
// video.setBitrate(90000);

let track = peerConnection.addTrack(video);
let session = new nodeDataChannel.RtcpReceivingSession();

track.setMediaHandler(session);

var dc = peerConnection.createDataChannel('datachannel_from_server');
track.onMessage((msg) => {
    dc.sendMessageBinary(msg);
});

Any hint what the problem is? Or some example with this feature?

Thank you and best regards Remigius

[paullouisageneau]

The issue is that by default the track receives data as RTP packets, which you seemingly send in the data channel, append together, and then try to decode as a VP8 file on reception. For this to work, you would need to depacketize RTP messages before sending them. Such a component does not exist in libdatachannel for now: https://github.com/paullouisageneau/libdatachannel/issues/676

However, I don't understand what you try to achieve here. If the goal is to implement a real-time video chat, you must not use a data channel like this as latency won't be low enough, instead you should use a media track with VP8 codec.

[murat-dogan]

I am closing the issue. Feel free to open a new issue when you need to. Thanks.