Can't change raw resolution in Bookworm

[Manuel-Angel-Es]

Discussed in https://github.com/raspberrypi/picamera2/discussions/838

^{Originally posted by **Manuel-Angel-Es** October 29, 2023} I recently installed Raspberry Pi Os Bookworm on my RPi4. I use picamera2 on a Super8 film scanner. With the scanner software it is possible to capture jpg and raw-dng images. With the Bullseye version, picamera2 worked correctly, as I expected. However, with Bookworm I run into an unexpected problem: I perform an initial configuration of the camera without problems, but when I try to change the capture resolution, only the resolution of the main stream changes, but not the resolution of the raw stream, which always remains constant. I would appreciate any suggestions to solve the problem. **This is picamera2's response after initially setting the camera to the resolution of 4056x3040 px:** 2023-10-29 10:14:23 - INFO - Camera configuration has been adjusted! [1:24:52.478260604] [71189] INFO Camera camera.cpp:1213 configuring streams: (0) 4056x3040-RGB888 (1) 4056x3040-SGRBG12_CSI2P [1:24:52.478806762] [71204] INFO RPI vc4.cpp:549 Sensor: /base/soc/i2c0mux/i2c@1/imx477@1a - Selected sensor format: 4056x3040-SGRBG12_1X12 - Selected unicam format: 4056x3040-pgCC 2023-10-29 10:14:23 - INFO - Configuration successful! **And below is picamera2's response after changing to the 2028x1520 px resolution:** 2023-10-29 10:16:04 - INFO - Camera configuration has been adjusted! [1:26:33.414200192] [71207] INFO Camera camera.cpp:1213 configuring streams: (0) 2028x1520-RGB888 (1) 4056x3040-SGRBG12_CSI2P [1:26:33.453206976] [71204] INFO RPI vc4.cpp:549 Sensor: /base/soc/i2c0mux/i2c@1/imx477@1a - Selected sensor format: 4056x3040-SGRBG12_1X12 - Selected unicam format: 4056x3040-pgCC 2023-10-29 10:16:04 - INFO - Configuration successful! **The following is the code used that worked correctly in Bullseye but does not work in Bookworm.** The change in resolution is made by calling the setSize(self, idx) function. With idx = 0 2028x1520 px is selected, with idx = 1 4056x3040 px is selected: from picamera2 import Picamera2, Metadata from time import sleep from logging import info class DS8Camera(): off = 0 previewing = 1 capturing = 2 # Resolutions supported by the camera. resolutions = [(2028, 1520), (4056, 3040)] def __init__(self): tuningfile = Picamera2.load_tuning_file("imx477_scientific.json") self.picam2 = Picamera2(tuning=tuningfile) # Configured resolution. self.resolution = self.resolutions[0] # Geometry of the zone of interest to be captured by the camera. # Coordinates of the upper left corner. self.x_offset = 0 self.y_offset = 0 # Zoom value. Determines the width and height of the zone of interest. # Values between 0.4 and 1. They are given by the zoom control. self.roiZ = 1 # Width of the image. self.width = self.resolution[0] * self.roiZ # Height of the image. self.height = self.resolution[1] * self.roiZ # Clipping rectangle (ScalerCrop). self.ScalerCrop = (self.x_offset, self.y_offset, self.width, self.height) # Automatic exposure. self.autoExp = False # Automatic white balance. self.awb = False # Number of bracketed exposures. self.bracketing = 1 # Stop points. self.stops = 0 # Manual exposure time. self.ManExposureTime = 2500 # Automatic exposure time. self.AeExposureTime = 2500 # Actual exposure time requested from the camera. self.exposureTime = 2500 # Minimum exposure time. It is fixed at 10 us. self.minExpTime = 10 # Maximum camera exposure time. # According to technical specifications of the camera, it can reach a # maximum of 670.74 s. # For our application we will set it to 1 s. self.maxExpTime = 1000000 # Metadata of the captured images. self.metadata = None # Camera capture speed in fps. self.frameRate = 10 # Camera settings. # These settings are applied with the camera disabled. # It's not possible modify them with the camera active. # Allocate a single buffer. self.picam2.still_configuration.buffer_count = 1 # Flip the image vertically self.picam2.still_configuration.transform.vflip = True self.picam2.still_configuration.transform.hflip = False # No images in preview. self.picam2.still_configuration.display = None # No streams are encoded. self.picam2.still_configuration.encode = None # Color space. # This feature is automatically configured by Picamera2. # Noise reduction: # This feature is automatically configured by Picamera2. # Duration time of the frames. self.picam2.still_configuration.controls.FrameDurationLimits = (self.minExpTime, self.maxExpTime) # Dimensions of the captured image. self.picam2.still_configuration.main.size = self.resolutions[1] self.picam2.still_configuration.raw.size = self.resolutions[1] # Image format 24 bits per pixel, ordered [R, G, B]. self.picam2.still_configuration.main.format = ("RGB888") # Image raw format. self.picam2.still_configuration.raw.format = ("SRGGB12_CSI2P") # Capture jpg images. self.captureJpg = True # Capture raw images. self.captureRaw = False # Unknown parameters. # Default configuration. self.picam2.still_configuration.main.stride = None # self.picam2.still_configuration.framesize = None self.picam2.still_configuration.lores = None # Do not allow queuing images. # The captured image corresponds to the moment of the capture order. # To queue images the buffer_count parameter must be greater than 1. self.picam2.still_configuration.queue = False # Loading still image settings. self.picam2.configure("still") # Camera controls. These parameters can be changed with the # camera working. # AeEnable: # AEC: Automatic Exposure Control. # AGC: Automatic Gain Control. # False: Algoritm AEC/AGC disabled. # True: Algoritm AEC/AGC enabled. self.picam2.controls.AeEnable = False # This variable gives error "Control AEConstraintMode is not advertised # by libcamera". # However, with the camera started it can be referenced normally. # AEConstraintMode: # 0: Normal. Normal metering. # 1: Highlight. Meter for highlights. # 2: Shadows. Meter for shadows. # 3: Custom. User-defined metering. # self.picam2.controls.AEConstraintMode = 0 # AeExposureMode: # 0: Normal. Normal exposures. # 1: Short. Use shorter exposures. # 2: Long. Use longer exposures. # 3: Custom. Use custom exposures. self.picam2.controls.AeExposureMode = 0 # AeMeteringMode: # 0: CentreWeighted. Centre weighted metering. # 1: Spot. Spot metering. # 2: Matrix. Matrix metering. # 3: Custom. Custom metering. self.picam2.controls.AeMeteringMode = 0 # ExposureTime: value between 0 and 1000000 us self.picam2.controls.ExposureTime = 4000 # NoiseReductionMode: configuration parameter. # FrameDurationLimits: configuration parameter. # Saturation: value between 0.0 and 32.0. Default 1.0. self.picam2.controls.Saturation = 1.0 # Brightness: value between -1 and 1. Default 0.0. self.picam2.controls.Brightness = 0.0 # Contrast: value between 0.0 and 32.0. Default 1.0. self.picam2.controls.Contrast = 1.0 # ExposureValue: value between -8.0 and 8.0. Default 0.0. self.picam2.controls.ExposureValue = 0 # AwbEnable: # AWB: Auto white balance. # False: Algoritm AWB disabled. # True: Algoritm AWB enabled. self.picam2.controls.AwbEnable = True # AwbMode: # 0: Auto. Any illumant. # 1: Incandescent. Incandescent lighting. # 2: Tungsten. Tungsten lighting. # 3: Fluorescent. Fluorescent lighting. # 4: Indoor. Indoor illumination. # 5: Daylight. Daylight illumination. # 6: Cloudy. Cloudy illumination. # 7: Custom. Custom setting. self.picam2.controls.AwbMode = 0 # ScalerCrop: self.picam2.controls.ScalerCrop = (0, 0, 4056, 3040) # AnalogueGain: value between 1.0 and 16.0. self.picam2.controls.AnalogueGain = 1.0 # ColourGains: value between 0.0 and 32.0 self.customGains = (2.56, 2.23) self.picam2.controls.ColourGains = self.customGains # Sharpness: value between 0.0 and 16.0. Default 1.0. self.picam2.controls.Sharpness = 1.0 self.mode = self.off # Starting up the camera. self.picam2.start() sleep(1) # Initial settings. # zoomdial, roiUpButton, roiDownButton def setY(self, value): self.y_offset = value self.ScalerCrop = (self.x_offset, self.y_offset, self.width, self.height) self.picam2.controls.ScalerCrop = self.ScalerCrop # zoomdial, roiLeftButton, roiRightButton def setX(self, value): self.x_offset = value self.ScalerCrop = (self.x_offset, self.y_offset, self.width, self.height) self.picam2.controls.ScalerCrop = self.ScalerCrop # Camera settings. # awbBox def setAwbMode(self, idx): if idx < 7: self.awb = True self.picam2.controls.AwbEnable = self.awb self.picam2.controls.AwbMode = idx else: self.awb = False self.picam2.controls.AwbEnable = self.awb if idx == 0: mode = "auto" elif idx == 1: mode = "incandescent lighting" elif idx == 2: mode = "tungsten lighting" elif idx == 3: mode = "fluorescent lighting" elif idx == 4: mode = "indoor lighting" elif idx == 5: mode = "daylight" elif idx == 6: mode = "cloudy" elif idx == 7: mode = "custom lighting" elif idx == 8: mode = "manual" else: return info("Adjusted white balance " + mode) # blueGainBox, redGainBox def fixGains(self, idx, value): self.metadata = self.captureMetadata() if (idx == 0): gred = value gblue = round(self.metadata.ColourGains[1], 2) elif (idx == 1): gred = round(self.metadata.ColourGains[0], 2) gblue = value self.picam2.controls.ColourGains = (gred, gblue) sleep(0.2) info("Camera color gains: blue = " + str(gblue) + ", red = " + str(gred)) # Capture. # captureStartBtn def startCaptureMode(self): sleep(1) self.mode = self.capturing info("Camera in capture mode") # Advanced settings. # constraintModeBox def setConstraintMode(self, idx): self.picam2.controls.AeConstraintMode = idx if idx == 0: mode = "normal" elif idx == 1: mode = "highlight" elif idx == 2: mode = "shadows" else: return info("Adjusted auto exposure restriction " + mode) # exposureModeBox def setExposureMode(self, idx): self.picam2.controls.AeExposureMode = idx if idx == 0: mode = "normal" elif idx == 1: mode = "sort exposures" elif idx == 2: mode = "long exposures" else: return info("Adjusted auto exposure mode " + mode) # meteringModeBox def setMeteringMode(self, idx): self.picam2.controls.AeMeteringMode = idx if idx == 0: mode = "centre weighted" elif idx == 1: mode = "spot" elif idx == 2: mode = "matrix" else: return info("Adjusted auto exposure metering mode " + mode) # resolutionBox def setSize(self, idx): self.picam2.stop() self.picam2.still_configuration.main.size = self.resolutions[idx] self.picam2.still_configuration.raw.size = self.resolutions[idx] self.picam2.configure("still") self.picam2.start() if idx == 0: resol = "2028x1520 px" elif idx == 1: resol = "4056x3040 px" info("Camera resolution " + resol) # This function is used to capture the metadata of the images. def captureMetadata(self): metadata = Metadata(self.picam2.capture_metadata()) return metadata

[cpixip]

A short test program which shows the issue is the following code:

import os
import picamera2 as pc2
import time

useNewConfig = True

class PicameraTest:
    def __init__(self):
        self.picamera2 = pc2.Picamera2()

        self.rawModes = [
            {"size":(1332,  990),"format":"SRGGB10"},
            {"size":(2028, 1080),"format":"SRGGB12"},
            {"size":(2028, 1520),"format":"SRGGB12"},
            {"size":(4056, 3040),"format":"SRGGB12"}
        ]        

        config = self.picamera2.create_still_configuration()

        self.mode = 3
        config['buffer_count'] = 4
        config['queue']        = True
        config['raw']          = self.rawModes[self.mode]
        config['main']         = {"size":self.rawModes[self.mode]['size'],'format':'RGB888'}

        self.picamera2.configure(config)        

    def setMode(self, para):
        # stop the camera
        self.picamera2.stop()       
        time.sleep(0.2)

        if useNewConfig:
            config = self.picamera2.create_still_configuration(raw={})
        else:
            config = self.picamera2.camera_configuration()

        config['raw']  = self.rawModes[int(para)]
        config['main'] = {"size":self.rawModes[int(para)]['size'], 'format':'RGB888'}

        self.picamera2.configure(config)

        self.picamera2.start()
        print('Video mode is now: %d' % int(para))
        request = self.picamera2.capture_request()
        request.save_dng(str(int(para)) + ".dng")
        request.release()

    def stop(self):
        self.picamera2.stop()

def main():
    # Print OS version
    os_version = os.popen('cat /etc/os-release').read()
    print(os_version)

    cam_test = PicameraTest()
    time.sleep(2)
    for i in range(len(cam_test.rawModes)):
        cam_test.setMode(i)
        time.sleep(0.1)  

     # Test modes in reverse order
    for i in reversed(range(len(cam_test.rawModes))):
        cam_test.setMode(i)
        time.sleep(0.1) 

    cam_test.stop()
    print("Test completed!")

if __name__ == "__main__":
    main()

With the global flag useNewConfig = True, the program runs as expected; with the flag useNewConfig = False, the resolution stays at the 4k res. Tests were done on a RP3 with the orignal HQ sensor and CMA=320MB. The same program runs flawlessly on earlier OS-versions.

[cpixip]

That bug is still there today, after upgrading the RP3 to the latest version of Bookworm, that is, with picamera2 version 0.3.14.

With useNewConfig = True I get the following line (as expected):

[0:41:36.315546877] [7482]  INFO Camera camera.cpp:1181 configuring streams: (0) 1332x990-RGB888 (1) 1332x990-SBGGR10
[0:41:36.317038680] [7498]  INFO RPI vc4.cpp:608 Sensor: /base/soc/i2c0mux/i2c@1/imx477@1a - Selected sensor format: 1332x990-SBGGR10_1X10 - Selected unicam format: 1332x990-BG10

with the selected sensor format as requested, namely 1332x990-SBGGR10_1X10.

But with useNewConfig = False, the selected sensor format is always 4056x3040-BG12, which is wrong:

[0:43:15.344342869] [7702]  INFO Camera camera.cpp:1181 configuring streams: (0) 1332x990-RGB888 (1) 4056x3040-SBGGR12
[0:43:15.345149893] [7718]  INFO RPI vc4.cpp:608 Sensor: /base/soc/i2c0mux/i2c@1/imx477@1a - Selected sensor format: 4056x3040-SBGGR12_1X12 - Selected unicam format: 4056x3040-BG12

One way to circumvent this is to start always with a new base config. In this case however, all the other parameters of a config need to be buffered externally and set again for each mode switch. There are use cases where the selected transform, buffer_count and NoiseReductionMode for example should stay as they are, even if the camera performs a mode switch - that used to work flawlessly in the previous OS-version, but fails now.

[davidplowman]

Hi, thanks for reporting this.

Would it be possible to submit a test script in the form

from picamera2 import Picamera2
picam2 = Picamera2()
config = picam2.create_preview_configuration(...stuff...)
picam2.configure(config)
print(picam2.camera_configuration())

as that would make it easy to see the problem and saves me from having to understand longer bits of code.

I should add that there have been changes in libcamera as to how configuration happens. The way we used the raw stream to force the sensor configuration was regarded as a bit of a "bodge", and so you now have to configure the sensor directly. Picamera2 will normally figure out the sensor configuration from the raw stream that you specify, so as to maintain backward compatibility. But it's possible you're falling into a difference in the new way of doing things.

As I said in the forum post, there is updated documentation coming but unfortunately that's a bit of a 1000-mile journey at the moment. I also said I'd try and post some abbreviated guidance shortly, and I am trying to get back to that. In the meantime, anything that makes test cases really easy to follow is a big help. Thanks!

[Manuel-Angel-Es]

@davidplowman,

Thanks for your attention.

I have prepared a small python script to highlight the problem:

from picamera2 import Picamera2
picam2 = Picamera2()
config = picam2.create_still_configuration()
picam2.configure(config)

picam2.still_configuration.main.size=(2028, 1520)
picam2.still_configuration.raw.size=(2028, 1520)
picam2.configure("still")
print(picam2.camera_configuration())

picam2.still_configuration.main.size=(4056, 3040)
picam2.still_configuration.raw.size=(4056, 3040)
picam2.configure("still")
print(picam2.camera_configuration())

picam2.close()

Look at the output of the first configuration. Both the size of the main stream and the raw stream are correct at the resolution of 2028 x 1520 px

After the second resolution change, the resolution of the main stream has changed, which is correct at 4056 x 3040 px, but the resolution of the raw stream has not changed, which remains at 2028 x 1520 px.

Script output results:

[20:27:51.377945844] [235416]  INFO Camera camera_manager.cpp:284 libcamera v0.1.0+99-4a23664b
[20:27:51.441276356] [235426]  WARN RPiSdn sdn.cpp:39 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise
[20:27:51.444275165] [235426]  INFO RPI vc4.cpp:444 Registered camera /base/soc/i2c0mux/i2c@1/imx477@1a to Unicam device /dev/media1 and ISP device /dev/media2
[20:27:51.444360590] [235426]  INFO RPI pipeline_base.cpp:1142 Using configuration file '/usr/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml'
[20:27:51.461632142] [235416]  INFO Camera camera_manager.cpp:284 libcamera v0.1.0+99-4a23664b
[20:27:51.509979292] [235429]  WARN RPiSdn sdn.cpp:39 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise
[20:27:51.512743160] [235429]  INFO RPI vc4.cpp:444 Registered camera /base/soc/i2c0mux/i2c@1/imx477@1a to Unicam device /dev/media1 and ISP device /dev/media2
[20:27:51.512868621] [235429]  INFO RPI pipeline_base.cpp:1142 Using configuration file '/usr/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml'
[20:27:51.520676048] [235416]  INFO Camera camera.cpp:1181 configuring streams: (0) 4056x3040-BGR888 (1) 4056x3040-SBGGR12_CSI2P
[20:27:51.521241873] [235429]  INFO RPI vc4.cpp:608 Sensor: /base/soc/i2c0mux/i2c@1/imx477@1a - Selected sensor format: 4056x3040-SBGGR12_1X12 - Selected unicam format: 4056x3040-pBCC
[20:27:51.557560304] [235416]  INFO Camera camera.cpp:1181 configuring streams: (0) 2028x1520-BGR888 (1) 2028x1520-SBGGR12_CSI2P
[20:27:51.558109722] [235429]  INFO RPI vc4.cpp:608 Sensor: /base/soc/i2c0mux/i2c@1/imx477@1a - Selected sensor format: 2028x1520-SBGGR12_1X12 - Selected unicam format: 2028x1520-pBCC
{'use_case': 'still', 'buffer_count': 1, 'transform': <libcamera.Transform 'identity'>, 'display': None, 'encode': None, 'colour_space': <libcamera.ColorSpace 'sYCC'>, 'controls': <Controls: {'NoiseReductionMode': <NoiseReductionModeEnum.HighQuality: 2>, 'FrameDurationLimits': (100, 1000000000)}>, 'main': {'size': (2028, 1520), 'format': 'BGR888', 'stride': 6144, 'framesize': 9338880}, 'lores': None, 'raw': {'size': (2028, 1520), 'format': 'SBGGR12_CSI2P', 'stride': 3072, 'framesize': 4669440}, 'queue': True, 'sensor': {'bit_depth': 12, 'output_size': (2028, 1520)}}
[20:27:51.577172545] [235416]  INFO Camera camera.cpp:1181 configuring streams: (0) 4056x3040-BGR888 (1) 2028x1520-SBGGR12_CSI2P
[20:27:51.577701722] [235429]  INFO RPI vc4.cpp:608 Sensor: /base/soc/i2c0mux/i2c@1/imx477@1a - Selected sensor format: 2028x1520-SBGGR12_1X12 - Selected unicam format: 2028x1520-pBCC
{'use_case': 'still', 'buffer_count': 1, 'transform': <libcamera.Transform 'identity'>, 'display': None, 'encode': None, 'colour_space': <libcamera.ColorSpace 'sYCC'>, 'controls': <Controls: {'NoiseReductionMode': <NoiseReductionModeEnum.HighQuality: 2>, 'FrameDurationLimits': (100, 1000000000)}>, 'main': {'size': (4056, 3040), 'format': 'BGR888', 'stride': 12192, 'framesize': 37063680}, 'lores': None, 'raw': {'size': (2028, 1520), 'format': 'SBGGR12_CSI2P', 'stride': 3072, 'framesize': 4669440}, 'queue': True, 'sensor': {'bit_depth': 12, 'output_size': (2028, 1520)}}

In previous versions of picamera2 the resolution change worked normally.

Regards

[davidplowman]

Hi, thanks for the simple test case, that's very helpful.

There have indeed been some changes in how libcamera configures the sensor, and I talk about this a bit in the release notes on the forum. Mostly I've tried to hide the change from users, but I haven't done it properly for people using the picam2.still_configuration idiom for setting up the camera configuration.

I'll fix this shortly, but for now the best workaround is to put

picam2.still_configuration = picam2.create_still_configuration()

at the top when you start changing the configuration.

Once I have fixed the code, the correct procedure will be to do

picam2.still_configuration.sensor.output_size = (4056, 3040)

instead of

picam2.still_configuration.raw.size = (4056, 3040)

(you can do both, though setting raw.size would be unnecessary).

[Manuel-Angel-Es]

Hi @davidplowman,

Thank you very much for your attention and your kind response.

I hope it doesn't take you too much work to modify the code.

Best regards

[cdcformatc]

I have a very similar problem while trying to flip the image, and the posted workaround fixes the issue. I hope that since the solution is the same that means the necessary code changes are the same!

picam2 = Picamera2(camera)
transform = libcamera.Transform(hflip=True, vflip=True)
still_config = picam2.create_still_configuration(transform=transform)
picam2.configure(still_config)
print(picam2.camera_configuration())

output:

{'use_case': 'still', 'transform': <libcamera.Transform 'hvflip'>, 'colour_space': <libcamera.ColorSpace 'sYCC'>, 'buffer_count': 1, 'queue': True, 'main': {'format': 'BGR888', 'size': (4608, 2592), 'stride': 13824, 'framesize': 35831808}, 'lores': None, 'raw': {'format': 'SRGGB10_CSI2P', 'size': (4608, 2592), 'stride': 5760, 'framesize': 14929920}, 'controls': {'NoiseReductionMode': <NoiseReductionModeEnum.HighQuality: 2>, 'FrameDurationLimits': (100, 1000000000)}, 'sensor': {'bit_depth': 10, 'output_size': (4608, 2592)}, 'display': None, 'encode': None}

However after capturing an image it is not flipped, even though the 'transform' member in the output would indicate otherwise.

Applying the workaround in this thread and adding the line

picam2.still_configuration = still_config

results in a properly flipped image.

I am thankful that @Manuel-Angel-Es opened this issue and that @davidplowman posted the workaround! Thanks everyone, hope this issue can be fixed soon.

[davidplowman]

@cdcformatc There has been some stuff in libcamera to do with transforms that got broken. I think maybe this release might be the first one to pick up that broken stuff even though it happened quite some time ago? I think it might be principally the reporting that's got messed up, libcamera is confused about what rotation it reports back to us. I believe it to be fixed in mainline libcamera now, but unfortunately those fixes break Picamera2 so there's some integration work to do. Sigh. I'll look into it, but maybe let's not discuss it in this issue as it is not the same thing. Thanks!

[davidplowman]

Back to the original problem, the proposed fix is here: https://github.com/raspberrypi/picamera2/pull/850