❓`frame.image.imageFormat` in frame processor is `PRIVATE`(34). Is it intended?

[bglgwyng]

Question

I found the code below from MLKit's face detector failed with the following error message.

val image = InputImage.fromMediaImage(frame.image, frame.imageInfo.rotationDegrees)

Exception in HostFunction: java.lang.IllegalArgumentException: Only JPEG and YUV_420_888 are supported now

So I investigated the image format of the frame.image and it was PRIVATE(34). I've already observed that MLKit's face detector worked fine with RNVC V2. So I suppose that this is a change from RNVC v3.

I don't fully understand the description of PRIVATE format in the android official document linked, but it doesn't appear to be a commonly employed value. Is it a bug? Or did I miss something?

I tested it on a Galaxy S20.

What I tried

No response

VisionCamera Version

3.0.0

Additional information

• [X] I am using Expo
• [X] I have read the Troubleshooting Guide
• [X] I agree to follow this project's Code of Conduct
• [X] I searched for similar questions in the issues page as well as in the discussions page and found none.

[rocket13011]

same issue

[GaylordP]

Same issue :/

[jlsneto]

Same issue! Please @mrousavy help us.

[mrousavy]

What happens if you pass pixelFormat="rgb" or pixelFormat="yuv" to the <Camera> component? Does that work?

[jjbwii]

Hello,

‘yuv’ crash my app (some internal function… it seems when VideoPipeline.kt tries to transform surface to image)

‘rgb’ works for me but there is no image bytes to copy behind… I need this info in order to implement some image post-processing with opencv…

[rocket13011]

hello everyone, with this PR there are no more YUV worries,

Basically, the problem is that the OpenGL pipeline was RGB-only.

so this pipeline was abandoned and switched to ImageWriter.

https://github.com/mrousavy/react-native-vision-camera/pull/1789

[mrousavy]

Hey- yea this PR fixes the frame processor issue once and for all, but it breaks for video recordings. Not sure why, as the ImageWriter should be writing Frames perfectly fine to the MediaRecorder's surface- I need to investigate this; https://github.com/mrousavy/react-native-vision-camera/pull/1799

[kyle-bowden]

I tested this change(#1799)

I am using react vision camera version 3.0.0

Pixel format set as "yuv" for frameprocessor pixelFormat="yuv"

and I get this error now

com.facebook.jni.CppException: Compiling JS failed: 1:1:invalid empty parentheses '( )' Buffer size 3 starts with: 280a29 and has protection mode(s): rw-p at com.mrousavy.camera.frameprocessor.FrameProcessor.call(Native Method) at com.mrousavy.camera.core.VideoPipeline.onImageAvailable(VideoPipeline.kt:79)

It is failing in the VideoPipeline.kt on fp.call(frame)

[itujo]

Hey, I have the same issue, have you guys figured out how to solve that?

[jikelnema]

Same issue @mrousavy.

[mrousavy]

fixed with 3.2.0 :)

[mgcrea]

@mrousavy Is the format supposed to be something else now? In my frame processor using 3.2.2 I still get 34 eg. ImageFormat.PRIVATE.

edit: passing pixelFormat="yuv" does change the format.

[mrousavy]

@mgcrea yup you need to pass the pixel format, I could maybe explain this a bit better in the docs?