Closed max-mapper closed 9 years ago
I think this is probably easier than you think as an RTCPeerConnection
can definitely carry multiple streams. While simple-peer
doesn't make this entirely obvious, I think you should be able to call the internal _setupVideo
method with an audio only stream and these will both be sent across the wire.
Having a look at @feross's code I'm fairly certain that you will then see two stream
events triggered for the peer. This being the case, then it's over to you to just inspect the stream and determine whether it's an audio or video stream. Which is nice and easy to do with a stream, using the getAudioTracks
or getVideoTracks
method: http://www.w3.org/TR/mediacapture-streams/#widl-MediaStream-getAudioTracks-sequence-MediaStreamTrack
@DamonOehlman the one big snag is that when doing specifically desktop screen sharing it is not allowed by Chromium to also send audio over the same stream
Ah, that's a shame. What error is reported? I might try and replicate this myself...
its a permission denied error, this one works:
var constraints = {
audio: false,
video: {
mandatory: {
chromeMediaSource: 'screen',
maxWidth: 1280,
maxHeight: 720,
maxFrameRate: 15
},
optional: []
}
}
but this triggers the error:
var constraints = {
audio: true,
video: {
mandatory: {
chromeMediaSource: 'screen',
maxWidth: 1280,
maxHeight: 720,
maxFrameRate: 15
},
optional: []
}
}
Yeah, I'm talking about doing two getUserMedia requests:
{ audio: true, video: false}
Then add both streams to the peer connection (trust me it can take it).
@DamonOehlman ohhhh, very cool
Feel free to send a PR to make simple-peer
support multiple streams. I just haven't needed that feature yet so I didn't add it.
@feross I got it working just now using simple-peer like this:
dunno if this breaks any internal state though (adding two streams)
var sp = require('./')
var p1 = sp({trickle: false})
navigator.webkitGetUserMedia({video: false, audio: true}, function (audioStream) {
navigator.webkitGetUserMedia({video: true, audio: false}, function (videoStream) {
var p2 = sp({initiator: true, trickle: false})
p2._pc.addStream(audioStream)
p2._pc.addStream(videoStream)
p1.on('signal', function(p1sdp) {
p2.signal(p1sdp)
})
p2.on('signal', function (p2sdp) {
p1.signal(p2sdp)
})
}, onErr)
}, onErr)
p1.on('stream', function(s) {
var kind = s.getTracks()[0].kind
if (kind === 'audio') renderAudio(s)
if (kind === 'video') renderVideo(s)
})
function renderVideo(stream) {
var video = document.createElement('video')
video.src = window.URL.createObjectURL(stream)
video.autoplay = true
document.body.appendChild(video)
}
function renderAudio(stream) {
var audio = document.createElement('audio')
audio.src = window.URL.createObjectURL(stream)
audio.autoplay = true
document.body.appendChild(audio)
}
function onErr (err) {
console.log('rtc err', err)
}
audio support is out now in 1.3.1
@feross when implementing this in screencat just now i noticed that it doesnt work if you getUserMedia after you do new Peer(). you have to do all the getUserMediaing first and then call new Peer() and then peer._pc.addStream for all streams. Not sure if this is fixable somehow
Yeah, makes sense that it wouldn't work. All the streams have to be added upfront because when you do sp({ initiator: true })
that kicks off the connection process, including generating an offer (which contains information about the streams on the PeerConnection). We could add a peer.init()
function to only kick things off when you're ready, so you can create the peer upfront... but that's just sugar :)
There's also theoretically some way to renegotiate the connection if you add a stream later on, but I think it's complicated, so it's not supported in simple-peer
right now.
WebRTC in Firefox 38: Multistream and renegotiation https://hacks.mozilla.org/2015/03/webrtc-in-firefox-38-multistream-and-renegotiation/
you can't do a screensharing session + and audio session over the same stream, but you can do two separate streams. we should add this :D hard part would be bootstrapping the second session (sdps) over the first stream