In-App Voice Chat

[curran]

As a VZCode user, I want to be able to chat using audio with my colleagues.

Possibly use Dolby.io

[curran]

Some good sample apps

https://github.com/dolbyio-samples/comms-app-react-events

https://github.com/dolbyio-samples/comms-app-web-react-classroom

The overall idea:

• We want to enable people to do voice chat inside of VZCode.
• Dolby.io provides the libraries that can make that happen

[curran]

https://github.com/dolbyio-samples/comms-sdk-web-getting-started/blob/main/video-calls/final/index.html

[curran]

Actually, we may be able to do it without the Dolby stuff, using pure open source and standards.

Some thoughts from ChatGPT:

Adding audio chat to a web application can be achieved using WebRTC (Web Real-Time Communication), a technology that enables real-time communication in web browsers. Here's a basic outline of the steps you can take to add audio chat to your web application:

1. Set up a signaling server: This server is used to exchange metadata between peers (e.g., users) to establish a connection. You can use WebSocket or Socket.IO for signaling.

2. Create a web page for the chat: Design a simple HTML page with a button to start the audio chat and an area to display the audio controls.

3. Implement WebRTC in your application: Use JavaScript to implement WebRTC in your application. This involves the following steps:

   • Get user media: Use the navigator.mediaDevices.getUserMedia() method to request access to the user's microphone.

     navigator.mediaDevices.getUserMedia({ audio: true })
     .then(function(stream) {
      // Use the stream
     })
     .catch(function(err) {
      console.log('Failed to get local stream', err);
     });

   • Create RTCPeerConnection: Create an RTCPeerConnection object for each peer. This object manages the connection and media exchange.

     var peerConnection = new RTCPeerConnection();

   • Add the stream to the connection: Add the obtained audio stream to the RTCPeerConnection object.

     peerConnection.addStream(localStream);

   • Exchange offer and answer: Use the signaling server to exchange offer and answer between peers to establish a connection.

     peerConnection.createOffer()
     .then(offer => {
      return peerConnection.setLocalDescription(offer);
     })
     .then(() => {
      // Send the offer to the remote peer through the signaling server
     });

   • Handle ICE candidates: Listen for ICE candidates and exchange them between peers through the signaling server.

     peerConnection.onicecandidate = function(event) {
     if (event.candidate) {
      // Send the candidate to the remote peer through the signaling server
     }
     };

4. Set up audio elements: Use HTML <audio> elements to play the audio streams received from other peers.

5. Handle events: Implement event handlers for various events such as connection establishment, receiving a remote stream, and disconnection.

6. Test your application: Test your audio chat application in different browsers and network conditions to ensure its reliability and performance.

7. Deploy your application: Once you're satisfied with the functionality and performance of your audio chat application, deploy it to a web server for public access.

Adding audio chat to a web application requires a good understanding of WebRTC and real-time communication concepts. You may also need to consider security aspects such as encryption and authentication to protect the communication.

[curran]

Are there some good open source packages that help with this setup?

Yes, there are several open-source packages and libraries that can help you set up audio chat in your web application, making the process more straightforward. Here are some popular ones:

1. SimpleWebRTC: This is a straightforward library for setting up WebRTC-based audio and video chat. It handles the complexities of WebRTC and provides a simple API for creating peer connections, managing sessions, and handling media streams.

   GitHub: https://github.com/andyet/SimpleWebRTC

2. PeerJS: PeerJS simplifies WebRTC peer-to-peer data, video, and audio calls. It provides a complete, configurable, and easy-to-use peer-to-peer connection API, encapsulating the complexity of WebRTC.

   GitHub: https://github.com/peers/peerjs

3. Socket.IO: While not specifically for WebRTC, Socket.IO is a powerful JavaScript library for real-time web applications. It's commonly used for signaling in WebRTC applications to establish peer connections.

   GitHub: https://github.com/socketio/socket.io

4. Janus WebRTC Server: Janus is a WebRTC Server that provides functionalities like video/audio conferencing, streaming, and more. It can be used as a component in your application to handle WebRTC connections.

   GitHub: https://github.com/meetecho/janus-gateway

5. Mediasoup: Mediasoup is a WebRTC SFU (Selective Forwarding Unit) for Node.js that allows for building multi-party video conferencing applications. It's highly scalable and provides low latency audio and video communication.

   GitHub: https://github.com/versatica/mediasoup

When choosing a package, consider factors such as the specific requirements of your application, the level of community support, and the frequency of updates to the package. These libraries and servers can significantly reduce the amount of code you need to write and simplify the process of adding audio chat to your web application.

[curran]

Are there any libraries for this that work well with NodeJS and Express?

Yes, there are several libraries that work well with Node.js and Express for setting up audio chat using WebRTC. Here are some options:

1. PeerJS Server: PeerJS provides a server package that can be easily integrated with Express. It handles WebRTC signaling for you, making it easier to set up peer-to-peer audio connections.

   GitHub: https://github.com/peers/peerjs-server

   Example usage with Express:

   const express = require('express');
   const { ExpressPeerServer } = require('peer');
   
   const app = express();
   const server = app.listen(9000);
   
   const peerServer = ExpressPeerServer(server, {
    path: '/myapp'
   });
   
   app.use('/peerjs', peerServer);

2. Socket.IO: Socket.IO is commonly used with Node.js and Express for real-time communication. It can be used for signaling in WebRTC applications to establish peer connections.

   GitHub: https://github.com/socketio/socket.io

   Example usage with Express:

   const express = require('express');
   const http = require('http');
   const socketIo = require('socket.io');
   
   const app = express();
   const server = http.createServer(app);
   const io = socketIo(server);
   
   io.on('connection', (socket) => {
    // Handle signaling data
    socket.on('signal', (data) => {
      // Broadcast signal to other peers
      socket.broadcast.emit('signal', data);
    });
   });
   
   server.listen(3000, () => {
    console.log('Server listening on port 3000');
   });

3. simple-peer: While not a server library, simple-peer is a lightweight wrapper around the WebRTC API that simplifies peer-to-peer connections. It works well with Node.js and can be used in conjunction with Socket.IO for signaling.

   GitHub: https://github.com/feross/simple-peer

   Example usage:

   const SimplePeer = require('simple-peer');
   const peer = new SimplePeer({ initiator: true, trickle: false });
   
   peer.on('signal', (data) => {
    // Send signaling data to the other peer via Socket.IO
   });
   
   peer.on('connect', () => {
    console.log('Peer connection established');
   });
   
   peer.on('data', (data) => {
    console.log('Received data from peer:', data.toString());
   });

These libraries can be integrated into your Node.js and Express application to handle the signaling and peer connection aspects of WebRTC, enabling you to add audio chat functionality to your web application.

All of this above is from ChatGPT.

[curran]

https://livekit.io/ was recommended. Might be perfect!

[curran]

Let's go with livekit.