💭 Use Vision Camera as Light Meter

sekhuat commented 3 weeks ago

Question

I would like to use the react-native-vision-camera to develop a Light Meter to measure the Exposure Value (EV) on the different part of the image captured by the device camera. I plan to use the frame processor from the library but the data pass into the frame variable only contain the image size, format and pixel values (returned by frame.toArrayBuffer()). The pixel values was useful but I need extra information like current ISO, shutter speed and aperture value use by the device camera that capture the frame. Only with all this information then I can calculate the EV in different area from the image.

Is it possible to return the current ISO, shutter speed and aperture value too in the frame object? Please advise, thank you.

What I tried

No response

VisionCamera Version

Ver 3

Additional information

[X] I am using Expo
[X] I have read the Troubleshooting Guide
[X] I agree to follow this project's Code of Conduct
[X] I searched for similar questions in the issues page as well as in the discussions page and found none.

bruno-centanaro commented 3 weeks ago

Is that information in the format or device object? If that's the case you can pass an extra object in each frame processor plugin call (the second param)

zzz08900 commented 3 weeks ago

AFAIK you can't have info like current ISO, WB point in preview, as frame processors work with preview images in almost real time. Also there's no shutter for preview frames.

What you probably need is take a photo with vision camera and somehow get all those params you need from the photo's metadata. Then you can read the photo from cache file or camera roll and get pixel values.

mrousavy commented 2 weeks ago

Hey - yea currently that is not something VisionCamera offers. This also shouldn't go into Frame Processors, as the purpose of Frame Processors is to really do some processing based on Frame pixel data, not camera live data, so here's your two options:

Manually analyze the Frame with some ML algorithms to find out how bright the pixels in the Frame are (this would work today, but might be slow and wouldn't give very precise results about EV, ISO etc)
Pay me to build that into VisionCamera - I can easily add callbacks for such things, e.g. onExposureChanged, onISOChanged, etc. as such things change over the runtime of the Camera, and I don't think they change so quick that we need a 60 FPS frame processor for them - I think Callbacks with new data every second or so are fine.

sekhuat commented 2 weeks ago

Hey - yea currently that is not something VisionCamera offers. This also shouldn't go into Frame Processors, as the purpose of Frame Processors is to really do some processing based on Frame pixel data, not camera live data, so here's your two options:

Manually analyze the Frame with some ML algorithms to find out how bright the pixels in the Frame are (this would work today, but might be slow and wouldn't give very precise results about EV, ISO etc)

Pay me to build that into VisionCamera - I can easily add callbacks for such things, e.g. onExposureChanged, onISOChanged, etc. as such things change over the runtime of the Camera, and I don't think they change so quick that we need a 60 FPS frame processor for them - I think Callbacks with new data every second or so are fine.

@mrousavy I just PM you via your email. Thanks.

mrousavy / react-native-vision-camera