search

l1npengtul / nokhwa

Cross Platform Rust Library for Powerful Webcam/Camera Capture
Apache License 2.0
533 stars 137 forks source link

yuv422 confusions? #146

Open yamt opened 1 year ago

yamt commented 1 year ago

this is about 0.10 branch as senpai is not buildable for me.

  1. nokhwa-bindings-macos seems to map both of kCMVideoCodecType_422YpCbCr8 and kCMPixelFormat_422YpCbCr8_yuvs to FrameFormat::YUYV. according to apple docs, kCMVideoCodecType_422YpCbCr8 is "ordered Cb Y'0 Cr Y'1" and kCMPixelFormat_422YpCbCr8_yuvs is "ordered Y'0 Cb Y'1 Cr." on the other hand, buf_yuyv422_to_rgb seems assuming the latter ordering.

  2. on my environment (MacBook Pro (15-inch, 2018)) the camera on the top of the display seems to report kCMPixelFormat_422YpCbCr8_yuvs. (confirmed by adding debug code in raw_fcc_to_frameformat) however, for some reasons, using the threaded-capture example, the buffers given to the callback actually seem to contain bytes in "Cb Y'0 Cr Y'1" order. (thus decoded wrong.)

cf. https://developer.apple.com/documentation/coremedia/1564239-video_codec_constants/kcmvideocodectype_422ypcbcr8 https://developer.apple.com/documentation/coremedia/1564244-video_pixel_format_constants/kcmpixelformat_422ypcbcr8_yuvs

l1npengtul commented 1 year ago

Thank you for your comment, I did not see that when I originally wrote that code. I will fix this in the upcoming release.

yamt commented 1 year ago

on my environment (MacBook Pro (15-inch, 2018)) the camera on the top of the display seems to report kCMPixelFormat_422YpCbCr8_yuvs. (confirmed by adding debug code in raw_fcc_to_frameformat) however, for some reasons, using the threaded-capture example, the buffers given to the callback actually seem to contain bytes in "Cb Y'0 Cr Y'1" order. (thus decoded wrong.)

https://github.com/l1npengtul/nokhwa/pull/151 fixes this part.