How to get texture from AR camera.

[meihkkkkkkk]

Hi, I'm trying to make an Android app to continuously get texture from AR Camera. The code is something like this: ` using System.Collections.Generic; using UnityEngine; using UnityEngine.UI; using UnityEngine.XR.ARFoundation; using UnityEngine.XR.ARSubsystems; using System; namespace UnityEngine.XR.ARFoundation.Samples { [RequireComponent(typeof(ARAnchorManager))] [RequireComponent(typeof(ARRaycastManager))]
public class AnchorCreator : MonoBehaviour { void Awake() {
m_RaycastManager = GetComponent(); m_AnchorManager = GetComponent(); m_Anchors = new List(); m_CameraBackground = GetComponent(); }

    void Update()
    {
        getTexture();
    }

    ARRaycastManager m_RaycastManager;
    ARAnchorManager m_AnchorManager;
    ARCameraBackground m_CameraBackground;

    public GameObject thing;
    public Text checktext;
    public RenderTexture renderTexture;
    public Texture2D tex;
    private void getTexture()
    {
        //Copy the camera background to a RenderTexture
        Graphics.Blit(null, renderTexture, m_CameraBackground.material);

        // Copy the RenderTexture from GPU to CPU
        var activeRenderTexture = RenderTexture.active;
        RenderTexture.active = renderTexture;

        tex = new Texture2D(renderTexture.width, renderTexture.height, TextureFormat.RGB24, true);
        tex.ReadPixels(new Rect(0, 0, renderTexture.width, renderTexture.height), 0, 0);
        tex.Apply();

        RenderTexture.active = activeRenderTexture;

        //This line is to display texture to a small image on screen
        thing.GetComponent<Image>().sprite = Sprite.Create(tex, new Rect(540, 960, 1000, 600), new Vector2(0.5f, 0.5f));
    }
}

} ` The code is modified from Anchor sample, so maybe some line are not nessessary. The problem is, when I run the app, it is supposed to be a full screen camera, with a small window (the Image in last code line) reflecting the camera. But that small window just shows notthing. Can you tell me if I miss something in my code? Thank you

[tdmowrer]

The code is modified from Anchor sample

Can you link me to that?

ReadPixels is pretty slow. I suggest using the XRCpuImage API instead. We have a sample for that.

[meihkkkkkkk]

Hi @tdmowrer , Firstly, thank you for replying so quickly. 1) > Can you link me to that? Well, it's the Anchor scene in the arfoundation-samples, posted by you. But my problem is not related to that. 2) I will learn about using XRCpuImage, but now I think speed is not what I'm concern about. 3) Let me specify my idea: Here's the screenshot of how the app work. The white box is attached to "GameObject Thing" in my code and it is supposed to show one part of the camera, but it shows notthing :( Do you have any sample related to my project? 4) Also, what I want to do is to get texture (like screenshots) continuously from AR camera, then use BarcodeReader.Decode to scan each texture to find qr code. Is that the right way? It's quite long but hope you can take a look and instruct me. Thank you so much

[tdmowrer]

Do you have any sample related to my project?

The CPU Images sample does something very similar.

what I want to do is to get texture (like screenshots) continuously from AR camera, then use BarcodeReader.Decode

See https://github.com/Unity-Technologies/arfoundation-samples/issues/489

[meihkkkkkkk]

Hi @tdmowrer , Thank you for replying. I have checked thread #489, the problem is, I cannot find some struct like:

• XRCameraImage, XRCameraImageConversionParams (I have add UnityEngine.XR.ARSubsystems, so these struct should be available, right?)

• CameraImageTransformation (I think it's in library named: UnityEngine.XR.ARExtensions, but I cannot find that library) Also, I have already download your repo, but can you show me where to find the sample which uses XRCameraImage?

if (cameraManager.TryGetLatestImage(out XRCameraImage image)) { // 'image' is an acquired resource that must be diposed when we are done. using (image) { // - Convert the image to RGBA. This takes considerable time, so there are also asynchronous // options. It's easier to understand the synchronous code for this example, though. // // - It's /much/ faster to convert to a single channel format, e.g., TextureFormat.R8, if // your 'BarcodeReader' can accept that. But if it only accepts an RGBA image, then this // should work too. var conversionParams = new XRCameraImageConversionParams(image, TextureFormat.RGBA32, CameraImageTransformation.MirrorY);

    // Get the size (number of bytes) of the buffer required to hold the RGBA pixel data.
    var dataSize = image.GetConvertedDataSize(conversionParams);
    var bytesPerPixel = 4; // because we chose an RGBA format.

    // Since the 'BarcodeReader' code you posted accepts a managed
    // array, that's what I've used here. It would be more efficient
    // to use a NativeArray if the BarcodeReader can use that.
    var pixels = new Color32[dataSize / bytesPerPixel];
    fixed (void* ptr = pixels)
    {
        image.Convert(conversionParams, new IntPtr(ptr), dataSize);
    }

    // The 'pixels' array now contains the image data.
    var result = barcodeReader.Decode(pixels, image.width, image.height);
}

}

[tdmowrer]

XRCameraImage, XRCameraImageConversionParams (I have add UnityEngine.XR.ARSubsystems, so these struct should be available, right?)

In 4.0, these were changed to XRCpuImage and XRCpuImage.ConversionParams.

CameraImageTransformation (I think it's in library named: UnityEngine.XR.ARExtensions, but I cannot find that library)

No -- ARExtensions was deprecated some time ago. See XRCpuImage.Transformation.

Also, I have already download your repo, but can you show me where to find the sample which uses XRCameraImage?

This description has links to the relevant scripts. The scene is simply called CpuImages.

[meihkkkkkkk]

XRCameraImage, XRCameraImageConversionParams (I have add UnityEngine.XR.ARSubsystems, so these struct should be available, right?)

In 4.0, these were changed to XRCpuImage and XRCpuImage.ConversionParams.

CameraImageTransformation (I think it's in library named: UnityEngine.XR.ARExtensions, but I cannot find that library)

No -- ARExtensions was deprecated some time ago. See XRCpuImage.Transformation.

Also, I have already download your repo, but can you show me where to find the sample which uses XRCameraImage?

This description has links to the relevant scripts. The scene is simply called CpuImages.

Hi @tdmowrer , I missed the CpuImages. Now it solves all my questions. Thank you so much for helping me all the time. Hope to learn more from you in the future