search

twilio / video-quickstart-ios

Twilio Video Quickstart for iOS
https://www.twilio.com/docs/api/video
MIT License
457 stars 177 forks source link

ARKit Example not working. Room getting a black screen from subscriber (participant) #582

Closed jbejas closed 2 years ago

jbejas commented 3 years ago

Description

When joining a room using the ARKit example app, the new subscriber added to the room, which is supposed to share the camera capture with AR object in it is black.

Steps to Reproduce

  1. Build ARKit Example and install it on a real device.
  2. Run the ARKit App in the device.
  3. Join an existing room (hardcoded in this case).
  4. The participants already in the room only get a black screen from ARKit example.

Code

The same as in the ARKit example. Just updated the Room name and accessToken.

Expected Behavior

ARKit example should share the camera and the AR Object being rendered on that screen and stream it to the desired room.

Actual Behavior

All participants in the desired Room only see a black screen being streamed from the ARKit example App

Reproduces How Often

Always.

Logs

Will provide logs soon.

Video iOS SDK

4.0.0 via CocoaPods

Xcode

12.2

iOS Version

14.0.0

iOS Device

iPad Mini 5, iPhone 12 Pro Max

ceaglest commented 3 years ago

Hi @jbejas,

Thank you for the report. I have a few follow up questions:

  1. Could you provide a Room SID where the Participant saw black video from the Publisher?
  2. Can you see the ARKit preview locally on the publisher side, or is it also black?

Thanks, Chris

jbejas commented 3 years ago

@ceaglest Here is the information you required.

Room SID: RM2d23e9234863fa10f5124161e805b51d

I can see the preview locally in the publisher. On the other side, I see a black screen. Attaching Screenshots.

image image

jbejas commented 3 years ago

@ceaglest just for clarification, the black screen you see in the first image, which is an ongoing call in the Room with the SID I provided to you, should show what you see in the second image, which is one of the participants in the same room. Best!

ceaglest commented 3 years ago

Hi @jbejas,

In RM2d23e9234863fa10f5124161e805b51d there is only one Participant and they are publishing their media to the Server since it is a Group Room. I don't see a second Participant connecting during the lifetime of the Room. Wherever your second Participant connected they did not join the same Room.

Can you issue a token for the second Participant and make sure you are connecting to the same Room named "TMRW POC"?

Best, Chris

jbejas commented 3 years ago

@ceaglest Sorry... I think I’ve sent you the wrong Room SID. Take a look at this one. RMa4e17a3928094e6dcbd4734081588244. Sorry again, and thanks for your help. Let me know if there is anything else I can do for this matter.

ceaglest commented 3 years ago

Hi @jbejas,

I looked at RMa4e17a3928094e6dcbd4734081588244. There are two unique Participants in this one, but they each connect twice. I see tracks subscribed and media flowing so I am not sure how to explain that you are only seeing a black screen.

What app do your viewers run, is this a modified version of VideoApp? It looks like perhaps you are not handling subscription events, or not adding renderers to remote video tracks?

To rule out your devices and the video SDK as the culprit, would it be possible to try our basic QuickStart example to subscribe to the ARKit Video? It is simple and can handle rendering one active video Track without any code changes.

First Round

  1. The first participant ("VIOS") connected with audio + video tracks. Some media was sent.
  2. The second participant ("ARKit Stream") connected with audio + video tracks.
  3. Both Participants subscribe to Tracks and media is flowing.

P2 starts reconnecting, is that because the app was killed? Media was flowing before this.

2020-12-29T16:04:29.470Z: [group_billing_event] for P1 with bytes sent 331552, bytes received 2049451, recording time in secs 0
2020-12-29T16:04:55.398Z: [group_billing_event] for P2 with bytes sent 903142, bytes received 292594, recording time in secs 0

Second Round

  1. P3 ("VIOS") connects with audio + video.
  2. P4 connects with audio + video.

P3 is doing a lot of publishing and unpunlishing of audio + enabling / disabling of video. Is this intended?

1  2020-12-29T16:06:18.847Z  track_added     MT5043e91b149167e2e19a4b17820ce6e9  FFDBae2a5d4c2fcABf9Ac7a995c0Feb7  video
    4  2020-12-29T16:06:48.562Z  track_disabled  MT5043e91b149167e2e19a4b17820ce6e9  FFDBae2a5d4c2fcABf9Ac7a995c0Feb7  video
    5  2020-12-29T16:06:50.293Z  track_enabled   MT5043e91b149167e2e19a4b17820ce6e9  FFDBae2a5d4c2fcABf9Ac7a995c0Feb7  video
    6  2020-12-29T16:06:59.696Z  track_disabled  MT5043e91b149167e2e19a4b17820ce6e9  FFDBae2a5d4c2fcABf9Ac7a995c0Feb7  video
    7  2020-12-29T16:07:01.526Z  track_enabled   MT5043e91b149167e2e19a4b17820ce6e9  FFDBae2a5d4c2fcABf9Ac7a995c0Feb7  video
    8  2020-12-29T16:07:25.810Z  track_disabled  MT5043e91b149167e2e19a4b17820ce6e9  FFDBae2a5d4c2fcABf9Ac7a995c0Feb7  video
    9  2020-12-29T16:07:27.221Z  track_enabled   MT5043e91b149167e2e19a4b17820ce6e9  FFDBae2a5d4c2fcABf9Ac7a995c0Feb7  video

ARKit user was sending and receiving media but also experienced an ice failure at the end. Was this because the app was hard killed?

2020-12-29T16:08:57.308Z: [group_billing_event] for P4 with bytes sent 8176785, bytes received 6366088, recording time in

Best, Chris