AVFoundation backend is not passing the frame format

l1npengtul / nokhwa

Cross Platform Rust Library for Powerful Webcam/Camera Capture

Apache License 2.0

531 stars 137 forks source link

AVFoundation backend is not passing the frame format #160

Open vyeevani opened 10 months ago

vyeevani commented 10 months ago

The pixelbuffer that's returned from the avfoundation callback is not guaranteed to be the same format as the one configured. This means that we have to query the cvpixelbuffer to get the pixel format that's being used by it for the decoder. The channel that's being used to pipe things from the callback to the poll API already has the pixel format but right now it's being hardcoded to grey. This should be dynamic.

marcpabst commented 10 months ago

Can you point me to the AVFoundation documentation? I think if a different pixel format is returned than requested, this should return an error to be handled by the user.

valenotary commented 3 months ago

@vyeevani coming across the same issue as you right now. On a 1920*1080 FaceTime HD cam on my Mac, I'm making requests for RGB but for some reason the frame buffer I get back seems to be some sort of two-channel pixel data instead of 3... every image decode fails because the resolution and buffer do not match dimensions.