Wrong image format received when retrieving CPUImage

[guerwan]

We are using ARFoundation on Unity and we are using the method: public bool TryAcquireLatestCpuImage(out XRCpuImage cpuImage)

We are converting the result to RGBA32 as explained here: https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@4.0/manual/cpu-camera-image.html But on some devices we get are still getting a BGR image instead of RGB.

Versions used:

• Unity 2019.3.2f1
• ARFoundation 4.2.3
• Example of device on which this issue happens: Xiaomi M2007J20CG - Android OS 12 / API-31 SKQ1.211019.001/V13.0.3.0.SJGEUXM

[guerwan]

We also noted a similar issue on another Xiaomi device:

• Xiaomi M2103K19PG - Android OS 11 On this device we are actually receiving a fully green image instead of a RGB or BGR image.

[guerwan]

Any update on this issue?

[guerwan]

Also the same issue on Redmi Note 11 Pro 5G, Android version 11. Any update would be very appreciated, thanks.

[tim-rohwedder]

I implemented the color conversion manually using OpenCV but experienced a similar problem. It appears that on some devices the U and V planes of the ArImage are swapped. In the documentation the order of planes is guaranteed to be YUV but on some devices this is definitely not the case.

We made sure to use all the functions described in the documentation we copy every plane separately into a corresponding Mat and take into account the pixel and row strides.

We observed this Bug on Redmi Note 10 Android version 12.

We also tried out the same code on various Samsung devices, here the problem does not occur and color channels are displayed correctly.

Maybe there is some caveat with Xiaomi devices that the ArCore implementation does not get right?

[tim-rohwedder]

Here is a code sample of how we implemented the conversion: colorConvSampleCode.txt

[guerwan]

I agree it seems to be an issue happening only on some Xiaomi devices.