search

And-Stuff / live-tools

A node.js based system for providing show critical services on XTV live broadcasts.
GNU General Public License v3.0
0 stars 0 forks source link

Intercom webRTC not working #5

Open standekker opened 5 years ago

standekker commented 5 years ago

The build in webRTC intercom is not working with the newer versions of browsers because of security issues and outdated pieces of code and techniques. As discussed with @Niwreg and @HansVanEijsden we are planning to take out all the old code assigned to the old intercom system and build a new intercom system with the newer webRTC standards

standekker commented 5 years ago

My idea was to use the mediaStream api like this and then use webRTC to establish a connection between the client and the server. Then the audio stream gets send over the webRTC connection from the user to the server. The server then uses one or more channelMergerNodes in order to add the incoming audio from multiple users to the same mix and then outputs the audio stream over webRTC back to the user to play the audio. This idea came from reading this article. Using this technique will enable us to make multiple communication channels, as well as enabling us to talk directly to one participant, in example, a presenter.

@Niwreg had the idea to set up a Asterix server in order to achieve this kind of communication. I do not have experience with Asterix, but also no hands on experience with webRTC and audiostreams in javascript. What do you guys @Niwreg @HansVanEijsden think is the best way to go?

HansVanEijsden commented 5 years ago

Asterisk is fine, just let me know if you need a test account. I can supply you with the username and password and I can enable all the necessary protocols. Currently I don't have much time and resources for code development, but it's no problem to create Asterisk accounts. Looks good to me!

standekker commented 5 years ago

Nice! I also have little time on my hands the upcoming weeks but i will see what i can do, i will contact you (@HansVanEijsden ) When i'm ready to start testing on a server.

Niwreg commented 5 years ago

My idea was to use the mediaStream api like this and then use webRTC to establish a connection between the client and the server. Then the audio stream gets send over the webRTC connection from the user to the server. The server then uses one or more channelMergerNodes in order to add the incoming audio from multiple users to the same mix and then outputs the audio stream over webRTC back to the user to play the audio. This idea came from reading this article. Using this technique will enable us to make multiple communication channels, as well as enabling us to talk directly to one participant, in example, a presenter.

@Niwreg had the idea to set up a Asterix server in order to achieve this kind of communication. I do not have experience with Asterix, but also no hands on experience with webRTC and audiostreams in javascript. What do you guys @Niwreg @HansVanEijsden think is the best way to go?

Does this work in node tho?

standekker commented 5 years ago

I tried to do a proof of concept with using socket.io to send a PCM audio stream from a client to the server, but i couldn't get it to work. Also haven't look at it for a while. I think it should work fine in node because the channelMergerNodes is part of the webAPI JS and accepts PCM streams to merge them, but i am not 100% sure about that

Niwreg commented 3 years ago

Hi Guys,

Maybe we can take this up for 2021?

standekker commented 3 years ago

Yeah that would be great, actually forgot about this project. Do we want to resurrect the full feature set? Voice, tally and ccg information interface?

Niwreg commented 3 years ago

Yes i think so not sure about the ccg part, let's see how if we can do this with nodecg so we can maybe use vmix /obs kind of applications. Interesting i saw https://peerjs.com/ maybe we can built around that, I think Voice/tally would be an first start

Hold your horses: https://github.com/nwah/peerjs-audio-chat