Using AR Camera and WebCamTexture issue

[Sterling-Malory-Archer]

What I want to do in my AR App is have the option for the user to take a picture with his front-facing camera having a filter over it.

I managed to get it working by using this code

` private bool camAvailable; private WebCamTexture frontCam; private Texture defaultBackground; string selectedDeviceName = "";

private Quaternion baseRotation;

public RawImage background;
public AspectRatioFitter fit;
// Start is called before the first frame update

void Start()
{
    defaultBackground = background.texture;
    WebCamDevice[] devices = WebCamTexture.devices;
    baseRotation = transform.rotation;

    if (devices.Length == 0)
    {
        camAvailable = false;
        return;
    }

    foreach (var camDevice in devices)
    {
        if (camDevice.isFrontFacing)
        {
            selectedDeviceName = camDevice.name;
            frontCam = new WebCamTexture(selectedDeviceName, Screen.width, Screen.height);
        }
    }

    if (frontCam == null)
    {
        Debug.Log("Unable to find front camera");
        return;
    }

    frontCam.Play();
    background.texture = frontCam;

    camAvailable = true;
}

// Update is called once per frame
void Update()
{
    if (!camAvailable)
    {
        return;
    }

    float ratio = (float)frontCam.width / (float)frontCam.height;
    fit.aspectRatio = ratio;

    float scaleY = frontCam.videoVerticallyMirrored ? -1f : 1f;
    background.rectTransform.localScale = new Vector3(1, scaleY, 1);

    int orient = -frontCam.videoRotationAngle;
    background.rectTransform.localEulerAngles = new Vector3(0, 0, orient);
    //transform.rotation = baseRotation * Quaternion.AngleAxis(frontCam.videoRotationAngle, Vector3.back);
}`

However, when I try to use it in conjunction with ARCore / ARKit the ARCamera goes completely black and doesn't seem to do anything.

Is there a way to combine these two options or not?

Any help is kindly appreciated.

I can provide a video further showcasing the issue if needed.

[tdmowrer]

You can get the image through ARFoundation using this API. There's a sample in this repo that shows how to use it, too.

[Sterling-Malory-Archer]

What I want to do is swap to Front-facing camera and then when the users clicks on a back button, I want to enable AR Detection, not get the image with back camera which I use for AR Detection

[tdmowrer]

Let me restate your use case: You want to start with a non-AR experience using the front-facing camera using the WebCamTexture API, and then, in response to some event, enable AR using the front-facing camera?

If I understand you correctly, then you can do that by disabling the WebCamTexture and enabling an ARSession. ARFoundation 4.0 lets you explicitly choose a camera facing direction (it's an option on the ARCameraManager). Note it will depend on the device; iOS devices require a TrueDepth camera to enable the front-facing (we call it "user-facing") camera.

[Sterling-Malory-Archer]

Hi Tim, sorry if my question was a little bit confusing

Basically, how I have my application set up is that the user has AR Camera enabled to detect images which give some specific rewards (In this case the ability to take pictures using the front-facing camera with an image overlay so that it looks like a filter) and then when they are done, I want to re-enable the AR Camera (which is the back camera).

Everything works up until a certain point where the AR Camera just goes black.

[tdmowrer]

You cannot use the WebCamTexture and AR at the same time. You should be be able to pause the ARSession, use the WebCamTexture, stop the WebCamTexture, and then re-enable the session. Or you can use the ARFoundation APIs to switch camera facing direction and the TryGetLatestImage API I linked to above.

Everything works up until a certain point where the AR Camera just goes black.

Can you provide more details on this? Exactly when does it go black? What platform? Is there any log output?

[Sterling-Malory-Archer]

I will try this first solution you provided.

Thank you so much!

[Sterling-Malory-Archer]

Hi Tim,

What I tried to do was, when the user clicks the button for filters, the AR Session Origin and AR Session GameObjects get .SetActive(false). And then when the user clicks on a return button to turn off the filter canvas (which in turn stops the WebCamTexture), I set AR Session Origin and AR Session to .SetActive(true).

But that still doesn't work.

[mdurand42]

Thanks @Sterling-Malory-Archer. We will investigate further and get back to you when we can.

[DavidMohrhardt]

@Sterling-Malory-Archer hello, sorry for the hold up on this issue. I took the time to put together a test case and I was unable to recreate the issue. I want to make sure that what I tested and what you are attempting to do are aligned so if I could get a little more information it would help.

First, can you tell me what the version of the packages you are using as well as the Unity version? Particularly, AR Foundation, AR Subsystems, ARCore, ARKit, and ARKitFaceTracking.

Second, what devices have you seen this behaviour on?

Finally, would you still be able to provide a short video of the behaviour?

[DavidMohrhardt]

Closing this as not reproducible, if you still have the issue @Sterling-Malory-Archer please feel free to reopen.