TwilioVideoParticipantView: Black Screen on Android only

[vyao95]

Steps to reproduce

1. Initiate a video call from android app
2. Answer call from web app
3. Web participant's video track triggers onParticipantAddedVideoTrack in android app
4. Render web participant's video track in android app => Black Screen

Expected behaviour

Once video/audio tracks are added by web participant, TwilioVideoParticipantView should render the video/audio track in the android app.

Actual behaviour

Remote video doesn't show, only a black screen. Remote audio comes through.

Code

const HelpDeskVideoCall = ({navigation: {navigate}}) => { const reservationInfoRef = useRef({}); const [ccsToken, setCCSToken] = useState(null); const [ccsResponse, setCCSResponse] = useState(null); const [status, setStatus] = useState(CallState.Connecting); const twilioRef = useRef(); const [videoTracks, setVideoTracks] = useState(new Map()); const user = useSelector(state => state.user); const { token } = useSelector(state => state.twilio); const [roomCreated, setRoomCreated] = useState(false);

const getAccessToken = async () => { try { const accessToken = await Api.getVideoAccessToken({ roomName: user.roomName, identity: 'complete-call-solutions' });

  const ccsResponse = await Api.startVideoCall({
    id: user.roomName,
    first_name: user.barcodeId ? user.barcodeId : user.appUserData.barcodeId,
    last_name: user.name ? user.name : user.appUserData.name,
    access_token: accessToken,
  });

  setCCSToken(accessToken);
  setCCSResponse(ccsResponse);
} catch (error) {
  console.log('twilio getAccessToken error===', error);
}

}

const handleConnect = () => { const event = { user_properties: { ...user.appUserData, barcodeId: user.barcodeId }, }; try { setStatus(CallState.Connecting);

  console.log('token get?', ccsResponse.agents_available);
  if (ccsResponse.agents_available === false) {
    event.event_type = "Twilio Connect Busy";
    setStatus(CallState.Busy);
  } else if (ccsResponse === 'undefine') {
    event.event_type = "Twilio Connect Maintenance";
    setStatus(CallState.Maintenance);
  } else {
    twilioRef.current &&
      twilioRef.current.connect({
        enableAudio: true,
        roomName: user.roomName,
        accessToken: token,
      });
    if(Platform.OS === 'ios') {
      twilioRef.current && twilioRef.current.setAudioSessionEnabled(true);
    }
    event.event_type = "Twilio Connect Success";
  }
  Api.logEvent(event);
} catch (error) {
  event.event_type = "Twilio Connect Failure";
  Api.logEvent(event);
  setStatus(CallState.Maintenance);
  console.log('twilio connect error===', error);
}

};

const handleDisconnect = () => { SocketClient.emit('leaveRoom', reservationInfoRef.current.TaskSid); ccsResponse && ccsResponse.session_id && Api.endVideoCall(ccsResponse.session_id); setRoomCreated(false); setStatus(CallState.Disconnected); };

const onRoomDidConnect = () => { console.log('onRoomDidConnect'); Api.logEvent({ event_type: "Help Desk Video Call: Twilio Room Connected" }); setRoomCreated(true); }; const onRoomDidDisconnect = ({roomName, error}) => { if(ccsResponse && ccsResponse.session_id === roomName) { console.log("onRoomDidDisconnect"); Api.logEvent({ event_type: "End Call" }); handleDisconnect(); } }; const onRoomParticipantDidDisconnect = () => { console.log("onParticipantDidDisconnect"); Api.logEvent({ event_type: "Help Desk Video Call: Twilio Participant Disconnected" }); // handleDisconnect(); }; const onRoomDidFailToConnect = () => { console.log("onParticipantDidDisconnect"); Api.logEvent({ event_type: "Help Desk Video Call: Twilio Room Failed To Connect" }); setStatus(CallState.Disconnected); }; const onParticipantAddedVideoTrack = ({participant, track}) => { console.log('some operator video joined?'); Api.logEvent({ event_type: "Help Desk Video Call: Operator Video Joined Call" }); setVideoTracks( new Map([ ...videoTracks, [ track.trackSid, {participantSid: participant.sid, videoTrackSid: track.trackSid}, ], ]), ); setStatus(CallState.Connected); }; const onParticipantRemovedVideoTrack = ({track}) => { console.log("onParticipantRemovedVideoTrack"); Api.logEvent({ event_type: "Help Desk Video Call: Twilio Participant Removed Video" }); videoTracks.delete(track.trackSid); setVideoTracks(new Map([...videoTracks])); // handleDisconnect(); };

const showEndCallBtn = useCallback(() => { if (status === CallState.Connecting) { return true; } return false; }, [status]);

useFocusEffect( useCallback(() => { getAccessToken(); }, []) );

useFocusEffect( useCallback(() => { if(ccsResponse) { console.log("handling connect"); handleConnect(); } Orientation.lockToLandscapeLeft(); return () => { console.log('out'); }; }, [ccsResponse]) );

return (

{status === CallState.Connecting && ( Someone will be with you shortly... )} {status === CallState.Connected && ( {Array.from(videoTracks, ([trackSid, trackIdentifier]) => { console.log(`Track Identifier: ${JSON.stringify(trackIdentifier)}`) return ( ); })} {count => ( Recording {formatRecordTime(count)} )} )} {showEndCallBtn() && ( )}

); };

export default HelpDeskVideoCall;

Logs

Logs: LOG handling connect LOG token get? undefined LOG onRoomDidConnect LOG onParticipantAddedVideoTrack LOG Track Identifier: {"participantSid":"PAe1c4e094b4f0d4e9df8af245d22ff0da","videoTrackSid":"MTc719eddf20de708a7572aee6b6a9c259"}

Environment

• Node.js version: v14.6.0
• React Native version: 0.61.5
• React Native platform + platform version: 11

react-native-twilio-video-webrtc

Version: npm version or "master"

[vyao95]

When I display all tracks in the room on web, all video/audio feeds are displayed correctly. Same applies to iOS. If I had to guess, I think that the TwilioVideoParticipantView is probably not loading the video feed quickly enough before the screen renders? Or maybe there's not enough bandwidth?

I got the TwilioVideoParticipantView to load the video feed correctly sometimes, but it fails most of the time

[slycoder]

Just spitballing but could this be related to a codec issue: https://www.twilio.com/docs/video/managing-codecs?

[vyao95]

It turns out there was a different issue. The screen was getting rendered multiple times because I used navigator.push() instead of just navigator.navigate(). Fixing that solved this issue on Android, but I'm not sure why that didn't cause problems on iOS.

Closing this.

[ombogdanJoinToIt]

same problem. how to fix this?