Upon some digging, we found out that when using RGB (or other 3 channel formats) 1/3 of the camera is zeroed out and when using RGBA (4 channels) 1/4 of the camera is zeroed out.
This was caused by incorrect usage of vImageConvert_Planar8toPlanarF, when in reality the buffers were interleaved, not planar.
Having assumed different memory layout, the converter function acted as if the image rows were N_CHANNELS times shorter in terms of bytes than in reality, resulting in that strange behavior.
If we convert manually, this doesnt happen. I used vDSP_vfltu8, vDSP_vsmul so things go fast.
mrousavy
commented
6 months ago
Fixing this: https://github.com/mrousavy/vision-camera-resize-plugin/issues/40
Upon some digging, we found out that when using RGB (or other 3 channel formats) 1/3 of the camera is zeroed out and when using RGBA (4 channels) 1/4 of the camera is zeroed out.
This was caused by incorrect usage of vImageConvert_Planar8toPlanarF, when in reality the buffers were interleaved, not planar.
Having assumed different memory layout, the converter function acted as if the image rows were N_CHANNELS times shorter in terms of bytes than in reality, resulting in that strange behavior.
If we convert manually, this doesnt happen. I used vDSP_vfltu8, vDSP_vsmul so things go fast.