"TryAcquireLatestImage" return an image with larger view than the image on ar camera background

[karahanoguzhan]

Hi, I am trying to detect hand with using ar foundation, I am getting frame using TryAcquireLatestImage and results don't match correctly. I have exactly the same problem as stated here: https://github.com/Unity-Technologies/arfoundation-samples/issues/476 But there is no solution provided there, so I am asking to get some help about this problem. Could you help me please? This is my screen shot:

[DavidMohrhardt]

Yes this is the normal behavior. Typically the provider takes a larger camera image then supplies some matrices to properly rotate and slice the pass through texture.

You may want to look into blitting or reading back the camera image: https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@5.0/manual/index.html#copying-the-camera-texture-to-a-render-texture-when-accessing-the-camera-image-on-the-gpu

[karahanoguzhan]

Thank you so much! Now I am able to match the points correctly. But I have a couple of questions here:

• To make my model work correctly, I need to access some information about the image provided by ArCameraManager. After many researches, I couldn't find any way to get these informations. These are:

1. How can I get the rotation of image provided by ArCameraManager?
2. How can I check if image provided is horizontally flipped?
3. How can I check if image provided is vertically flipped also?

Thanks for your help!

[DavidMohrhardt]

To answer your questions in order:

1. The rotation of the image is determined by the mounting of the camera on the actual device which can be determined from the ARCameraFrameEventArgs.displayMatrix property. You will need to extract the rotational axis from the matrix and determine what value theta to rotate the image by. Generally speaking probably easier to just apply the rotation to the raw image you are displaying.
2. (and technically 3.) Determining the flipped state is generally based on which camera is being used (user vs world). The camera image is taken from the view of the camera so with the world facing camera there is likely no need to do any transformations on the resulting image other than rotation. On the user facing camera, however, most AR applications treat it like a "mirror" so the image may need to be vertically flipped to mimic the "mirror" behavior after the image is captured.

[andyb-unity]

Closing this issue as asked and answered. Feel free to re-open if you have further questions.