[BUG] ColorFilterArrangement for HQ camera is wrong

[scriptorron]

For official HQ camera the camera properties report "RGGB" Bayer pattern (ColorFilterArrangement 0). The format names for the raw modes also start with "SRGGB". But HQ camera seems to have BGGR Bayer pattern (ColorFilterArrangement 3).

To Reproduce To show camera properties and raw modes:

from picamera2 import Picamera2
import pprint

cameras = Picamera2.global_camera_info()

for c, camera in enumerate(cameras):
    print(f'Camera {c}:')
    pprint.pprint(cameras[c])
    print()
    picam2 = Picamera2(c)
    print('Camera properties:')
    pprint.pprint(picam2.camera_properties)
    print()
    print("Raw sensor modes:")
    pprint.pprint(picam2.sensor_modes)

Console Output

[0:18:37.321340458] [3218]  INFO Camera camera_manager.cpp:297 libcamera v0.0.5+83-bde9b04f
[0:18:37.356820359] [3219]  INFO RPI vc4.cpp:437 Registered camera /base/soc/i2c0mux/i2c@1/imx477@1a to Unicam device /dev/media3 and ISP device /dev/media1
[0:18:37.356906376] [3219]  INFO RPI pipeline_base.cpp:1101 Using configuration file '/usr/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml'
Camera 0:
{'Id': '/base/soc/i2c0mux/i2c@1/imx477@1a',
 'Location': 2,
 'Model': 'imx477',
 'Rotation': 0}

[0:18:37.360313228] [3218]  INFO Camera camera_manager.cpp:297 libcamera v0.0.5+83-bde9b04f
[0:18:37.394701573] [3222]  INFO RPI vc4.cpp:437 Registered camera /base/soc/i2c0mux/i2c@1/imx477@1a to Unicam device /dev/media3 and ISP device /dev/media1
[0:18:37.394958957] [3222]  INFO RPI pipeline_base.cpp:1101 Using configuration file '/usr/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml'
Camera properties:
{'ColorFilterArrangement': 0,
 'Location': 2,
 'Model': 'imx477',
 'PixelArrayActiveAreas': [(8, 16, 4056, 3040)],
 'PixelArraySize': (4056, 3040),
 'Rotation': 0,
 'ScalerCropMaximum': (0, 0, 0, 0),
 'SystemDevices': (20750, 20751, 20738, 20739, 20740),
 'UnitCellSize': (1550, 1550)}

Raw sensor modes:
[0:18:37.408509365] [3218]  INFO Camera camera.cpp:1033 configuring streams: (0) 640x480-XBGR8888 (1) 1332x990-SBGGR10_CSI2P
[0:18:37.409158003] [3222]  INFO RPI vc4.cpp:565 Sensor: /base/soc/i2c0mux/i2c@1/imx477@1a - Selected sensor format: 1332x990-SBGGR10_1X10 - Selected unicam format: 1332x990-pBAA
[0:18:37.426352000] [3218]  INFO Camera camera.cpp:1033 configuring streams: (0) 640x480-XBGR8888 (1) 2028x1080-SBGGR12_CSI2P
[0:18:37.427140172] [3222]  INFO RPI vc4.cpp:565 Sensor: /base/soc/i2c0mux/i2c@1/imx477@1a - Selected sensor format: 2028x1080-SBGGR12_1X12 - Selected unicam format: 2028x1080-pBCC
[0:18:37.458063629] [3218]  INFO Camera camera.cpp:1033 configuring streams: (0) 640x480-XBGR8888 (1) 2028x1520-SBGGR12_CSI2P
[0:18:37.458973503] [3222]  INFO RPI vc4.cpp:565 Sensor: /base/soc/i2c0mux/i2c@1/imx477@1a - Selected sensor format: 2028x1520-SBGGR12_1X12 - Selected unicam format: 2028x1520-pBCC
[0:18:37.489310211] [3218]  INFO Camera camera.cpp:1033 configuring streams: (0) 640x480-XBGR8888 (1) 4056x3040-SBGGR12_CSI2P
[0:18:37.489903905] [3222]  INFO RPI vc4.cpp:565 Sensor: /base/soc/i2c0mux/i2c@1/imx477@1a - Selected sensor format: 4056x3040-SBGGR12_1X12 - Selected unicam format: 4056x3040-pBCC
[{'bit_depth': 10,
  'crop_limits': (696, 528, 2664, 1980),
  'exposure_limits': (31, None),
  'format': SRGGB10_CSI2P,
  'fps': 120.05,
  'size': (1332, 990),
  'unpacked': 'SRGGB10'},
 {'bit_depth': 12,
  'crop_limits': (0, 440, 4056, 2160),
  'exposure_limits': (60, 667244877, None),
  'format': SRGGB12_CSI2P,
  'fps': 50.03,
  'size': (2028, 1080),
  'unpacked': 'SRGGB12'},
 {'bit_depth': 12,
  'crop_limits': (0, 0, 4056, 3040),
  'exposure_limits': (60, 674181621, None),
  'format': SRGGB12_CSI2P,
  'fps': 40.01,
  'size': (2028, 1520),
  'unpacked': 'SRGGB12'},
 {'bit_depth': 12,
  'crop_limits': (0, 0, 4056, 3040),
  'exposure_limits': (114, 674191602, None),
  'format': SRGGB12_CSI2P,
  'fps': 10.0,
  'size': (4056, 3040),
  'unpacked': 'SRGGB12'}]

Expected behaviour 'ColorFilterArrangement': 2

Hardware : Pi 3 and Pi 4

Additional context "Debian GNU/Linux 11 (bullseye)"

libcamera and picamera2 installed with apt from distribution repositories:

libcamera-apps/stable,now 1.2.1-1 arm64 [installed]
libcamera-tools/stable,now 0~git20230720+bde9b04f-1 arm64 [installed]
libcamera0/stable,now 0~git20230720+bde9b04f-1 arm64 [installed,automatic]
python3-libcamera/stable,now 0~git20230720+bde9b04f-1 arm64 [installed,automatic]

python3-picamera2/stable,stable,now 0.3.12-2 all [installed]

[davidplowman]

Hi, thanks for the question. We're probably getting caught out by the camera rotation. Raspberry Pi cameras are all actually mounted upside down on the camera boards, so libcamera automatically applies a 180 degree rotation to undo this effect and turn the images the right way up. For some sensors - including all the Raspberry Pi ones - this rotates the Bayer pattern (e.g. BGGR becomes RGGB).

Whenever you request a raw image format, the Bayer order is in fact ignored, because you have no choice in the matter. It may take heed of the bit depth you attach (depending on what is available), and also whether you want packed or unpacked raw data. But it always just sets the Bayer order to the one and only permutation that is valid. Does that help?

[scriptorron]

I understand the effect of the camera rotation. In the new picamera2 version the Rotation property has changed from 180 to 0, but the data array still has the same orientation and Bayer pattern as in the version before.

Half year ago (older version of picamera2) I spend some time to find out the Bayer pattern in the data array. Picamera2 reported for the V1 camera the properties GBRG and Rotation 0. With experiments I proved the GBRG order in the data array. The HQ camera had properties RGGB and Rotation 180. Experiments shown a BGGR order in the data array. That made sense to me because a 180 degree rotation of RGGB makes BGGR.

With the new picamera2 version the HQ camera has properties RGGB and Rotation 0. But data is still BGGR. In contrast the V1 camera still has properties GBRG, Rotation 0 and data is still GBRG (I tried it right now).

Both cameras have now the same Rotation value. But for HQ I need to rotate the Bayer pattern (or the data array), for V1 not. The properties ColorFilterArrangement and Rotation are not enough to tell what the Bayer pattern in the data is. Likely this is not what you intended.

[davidplowman]

There was a recent-ish change down in libcamera itself where the meaning of the rotation property was changed. I believe that's going to be (possibly has been, in some repositories) changed back because it wasn't correct. But you should always be able to tell the correct Bayer format by looking at picam2.camera_configuration()['raw']['format'], I believe.

[scriptorron]

Thank you very much!

For HQ camera the picam2.camera_configuration()['raw']['format'] reports 'SBGGR12_CSI2P' while picam2.sensor_modes for same resolution and binning says 'SRGGB12_CSI2P'. That's a little bit confusing. But now I know which one relevant and I will change my software to look on the right one.

Many thanks for your quick and great support! If you like you can close this issue.