P13: Your car your way WebSockets and WebRTC Proof of Concept (PoC)

Description

This Proof of Concept (PoC) demonstrates the feasibility of real-time communication through an online chat and video-conference service, leveraging WebSockets and WebRTC technologies. The PoC aims to validate our proposal by showcasing a seamless exchange between users and customer service representatives, ensuring a robust and interactive user experience.

Why WebSockets & WebRTC?

WebSockets enable persistent, bidirectional communication between the client and server, allowing for instant message delivery and low-latency interactions. WebRTC facilitates peer-to-peer connections for audio, video, and data sharing, ensuring high-quality real-time communication without the need for intermediary servers. By developing this PoC, we aim to reassure stakeholders of the viability and effectiveness of our proposed online chat functionality.

Table of contents

• P13: Your car your way WebSockets and WebRTC Proof of Concept (PoC)
  • Description
  • Why WebSockets \& WebRTC?
  • Table of contents
  • Tech stack for the PoC
  • Front-End
  • Back-End
  • Pre-requisites
  • Node.js
  • Configuration
  • Node.js
  • Installation procedure
  • Front-End
  • Back-End
  • Features and Limitations
  • Features
  • Limitations
  • WebRTC glossary
  • WebRTC Peer-to-Peer Connection Setup
  • Get User Media (G.U.M.)
  • Create Peer Connection
  • Add Local Tracks to the Connection
  • Add Event Listeners to Peer Connection
  • ICE Candidate Event
  • ICE Candidate WebSocket Response
  • Sender (offerer)
    • User Wants to Initiate Call
    • Set Remote Description (Answer)
  • Receiver (answerer)
    • Set Remote Description (Offer)
    • Create Answer and Set Local Description
  • On Track Event
  • On Negotiation Needed Event
    • Frontend (FE)
    • Backend (BE)
    • Initiating Peer
    • Receiving Peer
  • Miscellaneous
  • Conclusion

Tech stack for the PoC

Front-End

• HTML
• SASS
• TypeScript
• Angular 18

Back-End

• Node.js 20.11.0
• ExpressJS 4.19.2 (for HTTP requests in this PoC to save time. Our Architecture Definition and Compliance docs specify Java + Spring Boot for production)
• Socket.io 4.7.5

Pre-requisites

Node.js

Install Node.js LTS to install dependencies and run both the Front-End and Back-End servers.

Configuration

Node.js

The project requires Node.js version 20.11.0

You can install Node.js also using the node version manager (nvm), after clicking on the latest release and installing the nvm-setup.exe file, you can upgrade or downgrade node version using the nvm commands

Here's the steps:

1. Check the list of node versions installed:

   nvm ls
   
   # Output example:
   # * 16.15.0 (Currently using 64-bit executable)

2. Install the node version to upgrade/downgrade to:

     nvm install 20.11.0

3. Use the freshly installed version:

     nvm ls
     # 20.11.0
     # * 16.13.1 (Currently using 64-bit executable)
   
     nvm use 20.11.0
     # Now using node v20.11.0 (64-bit)

In case it doesn't work:

Go to the nodejs/ folder and slightly rename it, then re-execute nvm use 20.11.0

Installation procedure

Cloning the project: To clone this repository from GitHub, run the following command: git clone https://github.com/LePhenix47/Lahouiti_Younes_P13_21062024 .

Front-End

1. Install the dependencies:

To start the Angular Front-End project, follow these steps:

• Navigate to the Front-End directory in your terminal:

cd front

• Install project dependencies using npm:

npm install

2. Starting the server

• After the dependencies are installed, you can start the development server by running:

npm run dev-https

Note: This project is compatible with other JS runtimes such as Bun. If you use one, you can simply run bun dev-https to start the server.

This command will compile the Angular application and start a development server in HTTPS (WebRTC requires a secure context to function correctly). You can then access the application in your browser at https://localhost:4200.

Back-End

1. Install the dependencies and configure .env file:

To start the NodeJS Back-End project, follow these steps:

• Navigate to the Back-End directory in your terminal:

cd back

• Install project dependencies using npm:

npm install

• Once the dependencies installed you'll need to create a .env file, with contents similar to the .env.sample file:

# Any random address is accepted
LOCAL_IP=

2. Starting the server

• After the dependencies are installed, you can start the development server by running:

npm start

# If you use the Bun JS runtime you can also use `bun start`

Note: This project is also compatible with alternative JS runtimes like Bun, in which case you can start the server with bun start.

This command will start the NodeJS development server. You can then access the API with: https://localhost:3000.

Features and Limitations

Features

This PoC includes the following features:

• Chat via WebSockets: A real-time chat system built using WebSockets, allowing instant communication between users.

• Video Conferencing with WebRTC: Peer-to-peer video conferencing implemented using WebRTC, enabling direct audio and video communication.

• Device Switching and Screen Sharing: WebRTC-based functionality to switch between different input devices (e.g., cameras, microphones) and the ability to share your screen.

• Screen Recording: Ability to record your screen, with options to preview the current recording, view a list of all recordings, and access detailed information (duration and size) and download buttons for each recording

Limitations

The PoC has some known limitations:

• Lack of Loading Status and Connection Feedback: There is no loading status displayed during local media switching or connection feedback during the WebRTC session, which may leave users uncertain about the connection status.

• Potential Memory Leak During Screen Recording: Starting a screen recording and then sharing the screen can cause a memory leak, potentially leading to performance issues.

• Buggy Room Handling: The room management system has a few bugs as well.

WebRTC glossary

Understanding the key terms and components in WebRTC is crucial for working with this PoC:

WebRTC Peer-to-Peer Connection Setup

This document outlines the process of setting up a peer-to-peer connection using WebRTC. The flow includes steps for both the sender and the receiver of the connection.

Get User Media (G.U.M.)

Use getUserMedia() to access the user's camera and microphone, and getDisplayMedia() to access the user's screen.

const ls1 = await navigator.mediaDevices.getUserMedia({ video: true, audio: true }); // User's webcam and microphone
const ls2 = await navigator.mediaDevices.getDisplayMedia({ video: true, audio: false  }); // User's screen

Create Peer Connection

Create an RTCPeerConnection object.

const pc = new RTCPeerConnection(stunTurnConfig);

Add Local Tracks to the Connection

Add local media tracks to the peer connection.

ls1.getTracks().forEach(track => pc.addTrack(track));
ls2.getTracks().forEach(track => pc.addTrack(track));

Add Event Listeners to Peer Connection

Listen for various events on the peer connection.

pc.onicecandidate = (event) => { /* Handle ICE candidates */ };
pc.ontrack = (event) => { /* Handle remote track */ };
pc.onnegotiationneeded = (event) => { /* Handle negotiation */ };
pc.onsignalingstatechange = (event) => { /* Debug signaling state */ };

ICE Candidate Event

When an ICE candidate is found, send it to the other peer via the signaling server.

socket.emit("ice-candidate", event.candidate);

ICE Candidate WebSocket Response

When receiving an ICE candidate from the signaling server, add it to the peer connection.

socket.on("ice-candidate", (candidate) => pc.addIceCandidate(candidate));

Sender (offerer)

User Wants to Initiate Call

Create an offer, set the local description, and send the offer via the signaling server.

const offer = await pc.createOffer();
await pc.setLocalDescription(offer);
socket.emit("offer", offer);

Set Remote Description (Answer)

When receiving an answer, set the remote description.

socket.on("answer", async (answer) => {
    await pc.setRemoteDescription(answer);
});

Receiver (answerer)

Set Remote Description (Offer)

When receiving an offer, set the remote description.

socket.on("offer", async (offer) => {
    await pc.setRemoteDescription(offer);
    const answer = await pc.createAnswer();
    await pc.setLocalDescription(answer);
    socket.emit("answer", answer);
});

Create Answer and Set Local Description

Create an answer, set the local description, and send the answer via the signaling server.

On Track Event

When a remote media track is added, handle it by creating a new MediaStream and adding the track to it.

pc.ontrack = (event) => {
    const remoteStream = new MediaStream();
    event.streams[0].getTracks().forEach(track => remoteStream.addTrack(track));
    remoteVideoElement.srcObject = remoteStream;
};

On Negotiation Needed Event

When negotiation is needed, handle it appropriately (e.g., create and send a new offer if necessary).

Frontend (FE)

• offer: Emitted when the initiating peer creates an offer.
• ice-candidate: Emitted when an ICE candidate is generated.
• answer: Emitted when the receiving peer creates an answer.

Backend (BE)

• offer: Broadcasts the SDP offer from the initiating peer to the receiving peer.
• ice-candidate: Broadcasts the ICE candidates between peers.
• answer: Broadcasts the SDP answer from the receiving peer to the initiating peer.

Initiating Peer

Outgoing Events:

1. Send SDP offer:

   socket.emit('offer', {
    type: 'offer',
    sdp: peerConnection.localDescription
   });

2. Send ICE candidates:

   peerConnection.onicecandidate = (event) => {
    if (event.candidate) {
      socket.emit('ice-candidate', {
        candidate: event.candidate
      });
    }
   };

Incoming Events:

1. Receive SDP answer:

   socket.on('answer', async (message) => {
    await peerConnection.setRemoteDescription(new RTCSessionDescription(message));
   });

2. Receive ICE candidates:

   socket.on('ice-candidate', async (message) => {
    try {
      await peerConnection.addIceCandidate(new RTCIceCandidate(message.candidate));
    } catch (e) {
      console.error('Error adding received ice candidate', e);
    }
   });

Receiving Peer

Outgoing Events:

1. Send SDP answer:

   socket.emit('answer', {
    type: 'answer',
    sdp: peerConnection.localDescription
   });

2. Send ICE candidates:

   peerConnection.onicecandidate = (event) => {
    if (event.candidate) {
      socket.emit('ice-candidate', {
        candidate: event.candidate
      });
    }
   };

Incoming Events:

1. Receive SDP offer:

   socket.on('offer', async (message) => {
    await peerConnection.setRemoteDescription(new RTCSessionDescription(message));
    const answer = await peerConnection.createAnswer();
   
    // Emit the answer, see "Send SDP answer"
   });

2. Receive ICE candidates:

   socket.on('ice-candidate', async (message) => {
    try {
      await peerConnection.addIceCandidate(new RTCIceCandidate(message.candidate));
    } catch (e) {
      console.error('Error adding received ice candidate', e);
    }
   });

Miscellaneous

💻 WebRTC flow chart

[View discussion here](https://github.com/LePhenix47/Lahouiti_Younes_P13_21062024/discussions/12)

Conclusion

This Proof of Concept (PoC) for "Your Car Your Way" successfully demonstrates the capabilities of WebSockets and WebRTC for real-time communication in an online chat and video-conference environment. Through the seamless integration of these technologies, we've showcased the potential to deliver an interactive, high-quality user experience that meets the expectations of modern digital communication.

The PoC validates the feasibility of implementing such technologies in a production environment, offering insights into both the advantages and challenges encountered during development. While the PoC highlights the robustness of WebSockets for instant messaging and WebRTC for video conferencing, it also sheds light on areas that require further refinement, such as connection feedback, memory management, and room handling.

By using Node.js for the Back-End and Angular for the Front-End, we were able to create a responsive and scalable architecture that aligns with current industry standards. The choice of these technologies also provided flexibility and ease of development, making this PoC not only a demonstration of the proposed solution but also a foundation for future enhancements.

In conclusion, this PoC serves as a strong foundation for the next stages of development, offering stakeholders a clear vision of the technical viability and user experience that can be expected in the final implementation. With further iteration and optimization, this approach has the potential to revolutionize how customers and service representatives interact in real-time, delivering a more personalized and efficient service experience.