Implement AR into Video call Android

[pareshdevatval]

Description

Hello I want to implement Twilio video calling with AR, I've tried to implement it into video-quickstart-android demo but at the receiver end, it's not showing anything into the window it shows the black screen. Following is the code that I have implemented. Please help.

public class CustomCapturerVideoActivity extends AppCompatActivity { private VideoView videoView; private Chronometer timerView; private LocalVideoTrack localVideoTrack; private View arFragment;

@RequiresApi(api = Build.VERSION_CODES.N)
@Override
public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_custom_capturer);
    videoView = findViewById(R.id.video_view);
    RelativeLayout rlAr = findViewById(R.id.rlAr);

    //arFragment = (ArFragment) getSupportFragmentManager().findFragmentById(R.id.ar_fragment);
    arFragment = findViewById(R.id.ar_fragment);

    // Once added we should see our linear layout rendered live below
    localVideoTrack = LocalVideoTrack.create(this, true, new ViewCapturer(rlAr));
    localVideoTrack.addRenderer(videoView);
}

@Override
protected void onDestroy() {
    localVideoTrack.removeRenderer(videoView);
    localVideoTrack.release();
    localVideoTrack = null;
    timerView.stop();
    super.onDestroy();
}

}

activity_custom_capturer.xml

<?xml version="1.0" encoding="utf-8"?> <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android" android:layout_width="match_parent" android:layout_height="match_parent" android:orientation="vertical">

<RelativeLayout
    android:id="@+id/rlAr"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:layout_weight="1">

    <fragment
        android:id="@+id/ar_fragment"
        android:name="com.google.ar.sceneform.ux.ArFragment"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:layout_weight="1" />

    <ImageView
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_centerInParent="true"
        android:src="@android:drawable/btn_dialog" />

</RelativeLayout>
<com.twilio.video.VideoView
    android:id="@+id/video_view"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:layout_weight="1" />

[zackm0571]

Hey @pareshdevatval thanks for reaching out. We don't currently validate the SDK with AR. In the future we plan on adding an AR quickStart. If you'd like you can zip up your project and I can investigate.

[pareshdevatval]

@Hey @zackm0571 Thanks for quick replay, You can find my Code here. Please investigate.

[zackm0571]

Hey @pareshdevatval,

Apologies for the late reply. I downloaded your project and was able to get the AR objects to render onto the black background by rendering the RelativeLayout instead of the AR Fragment, but was unable to get the camera to render. This may require a significant amount of research from our team.

Here is the code that got the AR objects rendering:

        localVideoTrack = LocalVideoTrack.create(this, true, new ViewCapturer(rlAr));

[Viokeoke]

I have the same issue, but the code works with 2.2.1 SDK version. On 3 and 4 doesn't works. @zackm0571 Did you update the android camera api from one to two?

[aaalaniz]

Hey everyone,

AR support is not currently on our roadmap. We appreciate the input, but I will close this issue for now.

Thank you