New Enhancement - Adding a phone participant

[bijmon]

I am trying to add a phone participant into the meeting. But my requirement is that the local participant can connect using a phone. So I am trying the remove the audio track of the local participant and publish the audio track of the phone participant to local participant. and at the same time I am not putting the phone participant in the participants sate. It will throw error while I publish the phone participant's audio track in to the local participant because it's not a local audio track. Do you have any suggestion how to tackle it? Please help. Thanks.

[philnash]

Hey @bijmon, how are you getting the audio track of the phone participant?

[bijmon]

@philnash Below is my code that I use in a backend service to connect a room and then through the call back endpoint I update a Sync Doc for the status of the call. At the client side participant gets connected through the participantConnected callback. Through the sync Doc at the client side I get the status of the calls until I get connected. Once I get connected I want to replace the audio track of the phone participant with the audio track of the local participant. Not sure if that's feasible. Or do you see any other way of doing? Thanks for your help.

const Twilio = require('twilio');

const dial = async ({ to, meetingId, envKeys }) => {
  const {
    studioFlowId: from,
    twilioAccountSid,
    twilioAuthToken,
    videoServiceUrl,
    twimlId,
  } = envKeys;
  const client = new Twilio(twilioAccountSid, twilioAuthToken);
  const callResult = await client.calls.create({
    url: `https://handler.twilio.com/twiml/${twimlId}?roomid=${meetingId}`,
    machineDetection: 'Enable',
    timeout: 20,
    method: 'GET',
    statusCallback: `${videoServiceUrl}/phonecallback?meetingid=${meetingId}`,
    statusCallbackEvent: ['initiated', 'answered', 'ringing', 'completed', 'in-progress', 'busy', 'no-answer'],
    statusCallbackMethod: 'POST',
    to,
    from,
  });
  console.log('New phone call started...');
  console.log('callResult:', callResult);
  const { VoiceResponse } = Twilio.twiml;
  const response = new VoiceResponse();
  const connect = response.connect();
  connect.room(meetingId);
  return callResult;
};

module.exports = {
  dial,
};

[philnash]

Ok, so you have your participant call into the video room from their browser and also dial in via a phone?

You have connected the phone audio to the room correctly, using <Connect> and <Room>.

I don't think I understand what you mean when you say you want to replace the audio track though. When you connect to a video room you can connect with audio and video tracks from the device the users is connecting on. When you connect to a video room via <Connect> you are creating a new participant with their own audio track. You can't merge these participants within the system.

You could still join up the browser participant and phone participant in the user interface though. If you knew that a user dialling in was connected to a participant already in the room (because the phone number matched something in your database, for example) then you could set the participant identity of the phone call to something similar to your existing participant's identity, maybe by suffixing "-phone". For example, if your browser participant had the identity "bijmon" then you could set the phone participant identity to "bijmon-phone".

Once you've done that, you could then merge those identities and tracks in your interface by looking for matching identities without the "-phone" suffix.

Is that the sort of thing you are trying to achieve?

[bijmon]

@philnash Yes. You got it right. We are giving option to join through phone for the participant. Once it's connected through the phone we want to disable the audio track that was connected through the video call. Basically we want to merge them for the audio. Yes we can put proper identity for each of them. We can get the handle of them. Can you please suggest how to merge them? And one more thing we don't want to display the phone participant in the app. Thanks for your help.

[philnash]

Can you please suggest how to merge them? And one more thing we don't want to display the phone participant in the app.

When I say "merge" the two participants, I really just mean that you don't display a phone participant and allow their audio to take over. So when you figure out that a phone participant has joined (because the identity ends with "-phone", for example) you would need to do a couple of things:

• For the local participant that matches the identity, you would disable their audio (participant.audioTracks.forEach(pub => pub.track.disable()).
• You would not render anything for the phone participant
• You could add the audio track of the phone participant to whatever you are rendering for the browser participant that matches

You would also need to consider some other cases.

• What happens if a phone participant joins and there is no browser participant that matches them?
• What happens if the browser participant leaves the call but the phone participant remains?
• What happens if the phone participant leaves the call but the browser participant remains?

Treating two participants as one like this is going to be quite a headache! I wish you well with it.

[bijmon]

@philnash Got your point. I decided to make it another participant and hide it through styling. Originally I didn't want to handle multiple participants. Because in the app we wanted to handle just two participants because of the mute/unmute show video/stop video and their notifications functionalities. Do you have a better idea how to handle them when there are more participants? Thanks you always for your quick response.

[philnash]

Yeah, there's no way to really "merge" a participant aside from in the UI.

I'm afraid I don't really have solutions for this, only questions like the ones above! I think you're going to have to work through this yourself to find the best approach that works for your application. Sorry I can't be of more help.

[bijmon]

@philnash I was asking about handling mute/unmute audio and video notifications with each other. Do you have any suggestion to handle them?

[philnash]

When a participant mutes/unmutes a track (enables/disables in terms of the API) then remote tracks will emit an "enabled" or "disabled" event. You can also listen on a RemoteParticipant object for "trackEnabled" and "trackDisabled" events.

How you handle those ties in tightly with how you are handling your participants in the UI.