medooze / media-server-node

WebRTC Media Server for Node.js
MIT License
802 stars 119 forks source link

Streaming / Broadcasting #140

Closed ueabu closed 4 years ago

ueabu commented 4 years ago

I am trying to use the media server to broadcast a video stream it receives from a client. I have a successful connection between the client the and the media server. My aim is to get the server to then broadcast the stream It is getting to any other device in the network but so far no luck. When I make a connection with broadcast (var ws = new WebSocket(url, "broadcast")) I get the message below:

[0x700007cb5000][1586391836.584][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:1665230601,rid:'']
[0x700007cb5000][1586391836.584][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [1665230601]
[0x700007cb5000][1586391836.778][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:2355286230,rid:'']
[0x700007cb5000][1586391836.778][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [2355286230]
[0x700007cb5000][1586391836.778][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:2355286230,rid:'']
[0x700007cb5000][1586391836.778][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [2355286230]
[0x700007cb5000][1586391836.778][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:2355286230,rid:'']
[0x700007cb5000][1586391836.778][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [2355286230]
[0x700007cb5000][1586391836.778][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:1665230601,rid:'']
[0x700007cb5000][1586391836.778][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [1665230601]
[0x700007cb5000][1586391836.778][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:2355286230,rid:'']
[0x700007cb5000][1586391836.811][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [2355286230]
[0x700007cb5000][1586391836.811][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:2355286230,rid:'']
[0x700007cb5000][1586391836.811][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [2355286230]
[0x700007cb5000][1586391836.811][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:2355286230,rid:'']
[0x700007cb5000][1586391836.811][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [2355286230]
[0x700007cb5000][1586391836.811][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:1665230601,rid:'']
[0x700007cb5000][1586391836.811][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [1665230601]
[0x700007cb5000][1586391836.811][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:1665230601,rid:'']
[0x700007cb5000][1586391836.951][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [1665230601]
[0x700007cb5000][1586391836.951][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:2355286230,rid:'']
[0x700007cb5000][1586391836.951][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [2355286230]
[0x700007cb5000][1586391836.951][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:2355286230,rid:'']
[0x700007cb5000][1586391836.951][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [2355286230]
[0x700007cb5000][1586391836.951][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:1665230601,rid:'']
[0x700007cb5000][1586391836.951][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [1665230601]
[0x700007cb5000][1586391836.951][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:1665230601,rid:'']
[0x700007cb5000][1586391836.953][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [1665230601]
[0x700007cb5000][1586391836.953][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:2355286230,rid:'']
[0x700007cb5000][1586391836.953][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [2355286230]

I tried looking into this https://github.com/medooze/media-server-node/blob/master/examples/streamer.js but still no luck. Can someone guide me on a way to set up streaming to other devices from the media server. I know the /svc/ receives a stream and sends it back to the person who sends the stream. I am trying to send the stream to the server and then have the server broadcast the stream to other device.

murillo128 commented 4 years ago

can you post the code you are using?

ueabu commented 4 years ago

Yes. On the client side, I have this

    initStream = ()  => {
        const url = "wss://"+window.location.hostname+":8000/";
        this.pc = new RTCPeerConnection();
        console.log(this.pc);
        var ws = new WebSocket(url, "broadcast");
        ws.onopen = () => {
            console.log("opened");
            console.log(url);
            navigator.mediaDevices.getUserMedia({
                audio: false,
                video: true
            }).then((stream) => {
                console.log(stream);
                this.gotStream(stream);
                //Add stream to peer connection
                this.pc.addStream(stream);
                return this.pc.createOffer();
            }).then((offer) => {
                console.log("create offer success");
                this.sdp = offer.sdp;
                this.pc.setLocalDescription(offer);
                console.log(this.sdp);
                ws.send(JSON.stringify({
                    cmd     : "OFFER",
                    offer       : this.sdp
                }));
                ws.send(JSON.stringify({
                    cmd     : "SELECT_LAYER",
                    spatialLayerId  : 1,
                    temporalLayerId : 2
                }));
            }).catch((error) => {
                console.log(error);
            });
        };

        ws.onmessage = (event) => {
            const msg = JSON.parse(event.data);
            console.log("The message answer");

            console.log(msg.answer);

            this.pc.setRemoteDescription(new RTCSessionDescription({
                type:'answer',
                sdp: msg.answer
            }), function() {
                console.log("JOINED");
            }, function (error) {
                console.error("Error joining", error);
            });
        }

        ws.onerror = (error) => {
            console.log("error", error)
        }
    };

And on the server side, I just cloned the repo from here: https://github.com/medooze/media-server-demo-node

I haven't changed anything so the index.js is still the same as well as all the js files in lib. The broadcast.js is this:

//Get the Medooze Media Server interface
const MediaServer = require("medooze-media-server");

//Get Semantic SDP objects
const SemanticSDP   = require("semantic-sdp");
const SDPInfo       = SemanticSDP.SDPInfo;
const MediaInfo     = SemanticSDP.MediaInfo;
const CandidateInfo = SemanticSDP.CandidateInfo;
const DTLSInfo      = SemanticSDP.DTLSInfo;
const ICEInfo       = SemanticSDP.ICEInfo;
const StreamInfo    = SemanticSDP.StreamInfo;
const TrackInfo     = SemanticSDP.TrackInfo;
const Direction     = SemanticSDP.Direction;
const CodecInfo     = SemanticSDP.CodecInfo;

//Create new streamer
const streamer = MediaServer.createStreamer();

//Create new video session codecs
const video = new MediaInfo("video","video");

//Add h264 codec
video.addCodec(new CodecInfo("h264",96));

//Create session for video
const session = streamer.createSession(video,{
    local  : {
        port: 5004
    }
});

module.exports = function(request,protocol,endpoint)
{
    const connection = request.accept(protocol);

    connection.on('message', (frame) =>
    {
        //Get cmd
        var msg = JSON.parse(frame.utf8Data);

        //Get cmd
        if (msg.cmd==="OFFER")
        {

            //Process the sdp
            var offer = SDPInfo.process(msg.offer);

            //Create an DTLS ICE transport in that enpoint
            const transport = endpoint.createTransport({
                dtls : offer.getDTLS(),
                ice  : offer.getICE() 
            });

            //Set RTP remote properties
            transport.setRemoteProperties({
                audio : offer.getMedia("audio"),
                video : offer.getMedia("video")
            });

            //Get local DTLS and ICE info
            const dtls = transport.getLocalDTLSInfo();
            const ice  = transport.getLocalICEInfo();

            //Get local candidates
            const candidates = endpoint.getLocalCandidates();

            //Create local SDP info
            let answer = new SDPInfo();

            //Add ice and dtls info
            answer.setDTLS(dtls);
            answer.setICE(ice);
            //For each local candidate
            for (let i=0;i<candidates.length;++i)
                //Add candidate to media info
                answer.addCandidate(candidates[i]);

            //Get remote video m-line info 
            let audioOffer = offer.getMedia("audio");

            //If offer had video
            if (audioOffer)
            {
                //Create video media
                let  audio = new MediaInfo(audioOffer.getId(), "audio");
                //Set recv only
                audio.setDirection(Direction.INACTIVE);
                //Add it to answer
                answer.addMedia(audio);
            }

            //Get remote video m-line info 
            let videoOffer = offer.getMedia("video");

            //If offer had video
            if (videoOffer)
            {
                //Create video media
                let  video = new MediaInfo(videoOffer.getId(), "video");

                //Get codec types
                let h264 = videoOffer.getCodec("h264");
                //Add video codecs
                video.addCodec(h264);
                //Set recv only
                video.setDirection(Direction.RECVONLY);
                //Add it to answer
                answer.addMedia(video);
            }

            //Set RTP local  properties
            transport.setLocalProperties({
                audio : answer.getMedia("audio"),
                video : answer.getMedia("video")
            });

            //Create new local stream with only video
            const outgoingStream  = transport.createOutgoingStream({
                audio: false,
                video: true
            });

            //Copy incoming data from the broadcast stream to the local one
            outgoingStream.getVideoTracks()[0].attachTo(session.getIncomingStreamTrack());

            //Get local stream info
            const info = outgoingStream.getStreamInfo();

            //Add local stream info it to the answer
            answer.addStream(info);

            //Send response
            connection.sendUTF(JSON.stringify({
                answer : answer.toString()
            }));

            //Close on disconnect
            connection.on("close",() => {
                //Stop
                transport.stop();
            });
        }   
    });
};

It doesn't seem to be doing any broadcasting or creating a stream. I thought it would be similar to https://github.com/medooze/media-server-node/blob/master/examples/streamer.js

murillo128 commented 4 years ago

The streamer demo is doing the reverse way, from rtp broadcasting to webrtc.

For how to setup the incoming stream from webrtc correctly check:

https://github.com/medooze/media-server-node/blob/master/manual.md

ueabu commented 4 years ago

The example in the document shows how to set the outgoing stream to the same client that sent the stream. There is no documentation on how to get the incoming stream and set it as outgoing stream/broadcast it to other clients that connect to the server

agouaillard-cosmo commented 4 years ago

you are right, there is no documentation.