search

node-webrtc / node-webrtc-examples

MediaStream and RTCDataChannel examples using node-webrtc
508 stars 161 forks source link

can we mix several incoming audio using RTCAudioSink and RTCAudioSource? #29

Closed ldenoue closed 4 years ago

ldenoue commented 4 years ago

Similar to the video compositing example, I am trying to grab incoming audio from several peerConnections and send back one RTCAudioSource to each connected client my mixing the audio data of each.

So if we have say 4 people connected, it would use RTCAudioSink to grab the audio samples of each, but then how do we merge all those samples into one?

Alternatively, could we add more than one RTCAudioTrack back to the incoming peerConnection(s) so that each would receive several audio tracks?

Laurent

numerized commented 4 years ago

Hello @ldenoue, have you found a solution to do this? Thanks

ldenoue commented 4 years ago

I understood that I could use a PCM audio mixer to mix all incoming audio samples and send its output to the connected clients but haven’t implemented it.

markandrus commented 4 years ago

@ldenoue

So if we have say 4 people connected, it would use RTCAudioSink to grab the audio samples of each, but then how do we merge all those samples into one?

One option is to mix them in your application code. This will not be very performant.

Alternatively, could we add more than one RTCAudioTrack back to the incoming peerConnection(s) so that each would receive several audio tracks?

Yes, you can. For every incoming RTCAudioTrack, add it as an outgoing RTCAudioTrack on the other RTCPeerConnections using addTrack.

ldenoue commented 4 years ago

@markandrus thanks for your comments. My understanding is that merging the audio samples will cost CPU on my server, while adding audio tracks so that each peerConnection transports N-1 audio tracks would cost bandwidth. Correct?