Problems, DataChannel and renegotiate

[SerRashin]

Hi Muaz Khan.

I decided to improve you code. I have one one question.

Two Users, connect with DataChannel. without audio or video, only for transmitting messages =)

Next I try to add a video here so.

connection = new RTCMultiConnection('<?=$id;?>');
                connection.open();
                document.querySelector('#audio_status').onclick=function(){
                    if(connection.mediaData.audio===false){
                        $(this).addClass('btn-danger');
                        connection.mediaData.audio=true;
                    }else{
                        $(this).removeClass('btn-danger');
                        connection.mediaData.audio=false;
                    }
                    connection.newAddStream({
                        audio:connection.mediaData.audio,
                        video:connection.mediaData.video,
                        screen:connection.mediaData.screen
                    });
                };

my new methods

window.RTCMultiConnection = function (channel) {
    ...
    ...
    /// you default code RTCMultiConnection object
    ...
    this.newAddStream=function(mediaData){
        rtcMultiSession.addStream(mediaData);
        connection.mediaData={
            audio:mediaData.audio,
            video:mediaData.video,
            screen:mediaData.screen,
            data:true
        };
    };
}

//// and new method in RTCMultiSession
window.RTCMultiSession=function(connection){
  ...
    /// you default code RTCMultiConnection object
    this.newAddStream=function(session){
            captureUserMedia(function(){
                for (var peer in connection.peers) {
                    var socket = connection.peers[peer].socket;
                    socket.send({recreateStream:true,session:session}); // it send to private socket in function 'socketResponse'. see socket responce below
                }
            },session);
        };
}

//// new conditions in socketResponse
function socketResponse(response) {
    if(response.userid==connection.userid)return;

    if(response.sdp){
        ....
    }
    if(response.recreateStream){
        log('recreateStream',response);
        peerConfig.session=response.session; // client is ready to receive new data
        PRIVATE_SOCKET.send({recreateOffer:true});// say broadcaster recreateOffer
    }
    if(response.recreateOffer==true){
        peer=new PeerConnection();
        peerConfig.attachStreams=connection.attachStreams;
        peer.create('offer',peerConfig);
    }
}

Result connection - http://clip2net.com/s/ij4Wch step 2. I turn on video. - http://clip2net.com/s/ij5dN0

After 5-7 sec dataChannel connection Crashed and viewer remove from conection.channels object

onclose:function(e){
    e.userID=_config.userID;
    connection.onclose(e);
    // suggested in #71 by "efaj"
    if (connection.channels[e.userID])
        delete connection.channels[e.userID];
},

variant 2 recreateOffer and recreateAnswer

this variant not work, (offerer recreate SDP wrong) the viewer gets Old SDP.

Is it possible to solve the problem of disconnection or realize dataChannel normal revision?

PS. my first option allows you to make renegotiate, can be used as a temporary solution for Mozilla =)

[SerRashin]

if broadcaster - Mozilla Firefox, and viewer - Google Chrome - dataChannel not crashed i.e.my way of revision work in chrome and firefox goood.

you way of revision it - recreateOffer and recreateAnswer

i.e.

/// offer
 this.connection.createOffer(function (sessionDescription) {
    self.connection.setLocalDescription(sessionDescription);
    self.onSessionDescription(sessionDescription, self.streaminfo); // send to answerer
}, this.onSdpError, this.constraints);

/// answer
  this.setRemoteDescription(this.offerDescription);
 this.connection.createAnswer(function (sessionDescription) {
    self.connection.setLocalDescription(sessionDescription);
    self.onSessionDescription(sessionDescription, self.streaminfo); // send to offerer
}, this.onSdpError, this.constraints);

/// offerer
setRemoteDescription(answerDescription);

it work only chrome, firefox not create new works SDP

/// offer
 this.connection = new RTCPeerConnection(this.iceServers, this.optionalArgument); // new
 /// ... createDataChannel
 /// ... onaddstream
 /// ... etc ...

 this.connection.createOffer(function (sessionDescription) {
    self.connection.setLocalDescription(sessionDescription);
    self.onSessionDescription(sessionDescription, self.streaminfo); // send to answerer
}, this.onSdpError, this.constraints);

/// answer
 this.connection = new RTCPeerConnection(this.iceServers, this.optionalArgument); // new
 /// ... createDataChannel
 /// ... onaddstream
 /// ... etc ...
  this.setRemoteDescription(this.offerDescription);
 this.connection.createAnswer(function (sessionDescription) {
    self.connection.setLocalDescription(sessionDescription);
    self.onSessionDescription(sessionDescription, self.streaminfo); // send to offerer
}, this.onSdpError, this.constraints);

/// offerer
setRemoteDescription(answerDescription);

it work in chrome and firefox =)

firefox has only one problem, break the old connection dataChannel

[muaz-khan]

If caller or callee is Firefox; RTCMultiConnection recreates "peer-connection" instead of merely recreating "session-descriptions".

The next remaining part in v1.8 is to make sure that "session" object has all "renegotiation" values when we are recreating peer connection.

We simply need to store all "sessions" from "addStream" invocations in a RTCMultiSession-level object which is a private object accessible among all RTCMultiConnection APIs. E.g.

// on constructor initialization
rtcMultiSession.allSessions = connection.session;

// when user invokes "addStream" method
connection.addStream = function (session) {
    rtcMultiSession.allSessions = merge(rtcMultiSession.allSessions, session);
    // ---
};

// now, when recreating peer connection
connection.session = rtcMultiSession.allSessions;
connection.peers['target-userid'].redial();

[SerRashin]

Another question is to remove the old Stream necessary to cause removal PEER.removeStream(); only answerer or offerer and answerer ?

[muaz-khan]

First of all, as far as I know, removeStream was not implemented in Firefox. Though, I don't know current support.

removeStream should be invoked for the peer wanting to remove his media; it can be caller or callee.

removeStream removes MediaStream if available in the array; otherwise it removes first available MediaStream.

[SerRashin]

@muaz-khan Thanks. I try to write your own version of the code. Share code tomorrow. Maybe it will help you improve your code.

[SerRashin]

hi @muaz-khan . Tell me please. Why use this line:

 if (isData(this.session) && isFirefox) {
                    navigator.mozGetUserMedia({
                        audio: true,
                        fake: true
                    }, function (stream) {
                        self.connection.addStream(stream);

                        if (type == 'offer') {
                            self.createDataChannel();
                        }

                        self.getLocalDescription(type);

                        if (type == 'answer') {
                            self.createDataChannel();
                        }
                    }, this.onMediaError);
                }

Why in the data channel include fake user media data? it need only for old version firefox, or for something else?

without this code dataChannel good work. interesting what is its purpose =)

[muaz-khan]

Ah, yes, I'm still using old dirty workaround; though it is fixed since a while (I think since mozilla24). I'll update v1.8 very soon.

[SerRashin]

@muaz-khan it very good=) i will be waiting =) tell me please. how to remove the existing stream? peer.removeStream(connection.streams[label].stream); not work in mozilla error -

Error: removeStream not yet implemented
peer.removeStream(connection.streams[label].stream);

is there a way in Firefox to remove old and add a new stream?

[SerRashin]

@muaz-khan Hi. function connection._getStream method stop, not work, if i call - > connection.streams.stop() from my application

connection._getStream = function (e) {
            return {
                .....
                stop: function (forceToStopRemoteStream) {
                    this.sockets.forEach(function (socket) {
                        // this.type == undifened

need to add 'self'

connection._getStream = function (e) {
            return {
                .....
                stop: function (forceToStopRemoteStream) {
                     var self =this; // new this
                    this.sockets.forEach(function (socket) {
                      //   self.type == local or remote - good=)

[SerRashin]

remove impossible ? https://github.com/muaz-khan/WebRTC-Experiment/issues/225#issuecomment-46121319

[muaz-khan]

For Firefox, until removeStream gets implemented, we need to reconstruct peer object to attach new media stream; and old peer and relevant connections/references will be released and garbage collected.

[SerRashin]

@muaz-khan hi. it will be enough? clean connestion.streams and connestion.attachStrems and recreate offerere and all answerer without all streams.

ie essentially rebuilding connections to the original view, without media streams ?

[SerRashin]

@muaz-khan Hi again =)

not work offerer stream.onended

function captureUserMedia(callback, _session) {
    ...
    function _captureUserMedia(forcedConstraints, forcedCallback, isRemoveVideoTracks, dontPreventSSLAutoAllowed) {
        ...
        var mediaConfig = {
            onsuccess: function (stream, returnBack, idInstance, streamid) {
                ...
                stream.onended = function () { //////////// NOT WORK IN MOZILLA

                }
            }
        }
    }
}

Solution of the problem. function connection._getStream method stop, add new if

connection._getStream = function (e) {
            return {
                .....
                stop: function (forceToStopRemoteStream) {
                    var self =this;
                    if(isFirefox)self.stream.onended(self.streamObject);

[muaz-khan]

Thanks @serhanters. Now I'm manually invoking stream.onended in the stopTracks function. You can test latest updates here:

• https://www.rtcmulticonnection.org/latest.js

function stopTracks(mediaStream) {
    if (!mediaStream) throw 'MediaStream argument is mandatory.';

    if (typeof mediaStream.getAudioTracks == 'undefined') {
        if (mediaStream.stop) {
            mediaStream.stop();
        }
        return;
    }

    if (mediaStream.getAudioTracks().length && mediaStream.getAudioTracks()[0].stop) {
        mediaStream.getAudioTracks()[0].stop();
    }

    if (mediaStream.getVideoTracks().length && mediaStream.getVideoTracks()[0].stop) {
        mediaStream.getVideoTracks()[0].stop();
    }

    //--------------------------------- here:
    if (isFirefox) {
        // todo-verify: this may cause multiple-invocation of "onstreamended"
        if (mediaStream.onended) mediaStream.onended();
    }
}

I also updated stop method:

stop: function (forceToStopRemoteStream) {
    var self = this;

    // .............

    var stream = self.stream;
    if (stream) stopTracks(stream); //---------------- this line
}

[SerRashin]

@muaz-khan if (isFirefox && session.data) {

you forgot to add 'this' or 'self' (line 2363)

[SerRashin]

@muaz-khan idea. dynamic reception audio&&video or audio||video

// renegotiate new stream
this.addStream = function (e) {
    .....
    function addStream(_peer) {
        var socket = _peer.socket;
        if (!socket) {
            warn(_peer, 'doesn\'t has socket.');
            return;
        }
        socket.send({newSession:session});
    }
}

// function SocketResponce
function socketResponse(response) {
    if (response.userid == connection.userid)return;
    if(response.newSession){
        connection.session=response.newSession;
        log(connection.session);
    }
    ................
}

it for Firefox (prepare answerer to receive stream)

[SerRashin]

@muaz-khan . I have question.

if i add stream (with video ) and stop stream (connection.streams.stop()) and then add stream (with video ) everything works fine.

but if i add stream (with video ) and stop stream (connection.streams.stop()) and then add stream (with AUDIO) nothing does not work.

see:

1. addStream({video:true}); - http://clip2net.com/s/ilEdtG
2. connection.streams.stop()
3. addStream({audio:true}); - http://clip2net.com/s/ilEput

answerer not receiving stream if switch the audio to video or video to audio.

why ? =(

[muaz-khan]

Simply set sdpConstraints with both OfferToReceive Audio/Video and it will work:

// you must set it before calling "open" or "join" method
connection.sdpConstraints.mandatory = {
    OfferToReceiveAudio: true,
    OfferToReceiveVideo: true
};

[SerRashin]

@muaz-khan yes, it work, but works only in Chrome. In Firefox, stream played, only if unless specify what data come

ie. example

stream (only audio) - set OfferToReceiveAudio in true stream (only video) - set OfferToReceiveVideo in true stream (only audio&&video) - set OfferToReceiveAudio&&OfferToReceiveVideo in true

IF set OfferToReceiveAudio&&OfferToReceiveVideo in true stream not work.

if use your method works only Chrome. if use my method https://github.com/muaz-khan/WebRTC-Experiment/issues/225#issuecomment-46571924 it work in Chrome and Firefox.