[HOW-TO] The preview window doesn't show the most recent frame

[daodats]

I'm using external trigger to let Global shutter camera to capture image, my setup using the preview configuration, but after the app launch, the preview window display don't show the most recent captured frame, it display the frame that right before the most recent one. So how can I let the preview window always show the newest frame?

[davidplowman]

Hi, are you able to tell whether this issue is specific to using an external trigger or does it happen with any capture? Is it possible for you to create the most minimal example (preferably just a few lines) that demonstrates the problem?

Could you please also confirm what kind of a Pi you are using, what OS you are using, and that all your software (including libcamera, Picamera2) is up to date. Thanks!

[daodats]

Hi @davidplowman, My setup includes: Board: Raspberry Pi 3B, OS: Bookworm build 2023/12/05 libcamera v0.1 Picamera2 v.0.3.17-(1)

Here my setup code: picam2 = Picamera2()

camera_config = picam2.create_preview_configuration(
        main={"size":(400, 280)}
    )

picam2.configure(camera_config)
picam2.set_controls({"ExposureTime": g_exposure_time, "Brightness": 0, "AeExposureMode": 1})

picam2.start_preview(Preview.QTGL, x=10, y=100, width=PREVIEW_WIDTH, height=PREVIEW_HEIGHT)
picam2.title_fields = ["ExposureTime", "AnalogueGain"]

picam2.start()

And I capture by the function: capture_array()

Thanks!

[davidplowman]

Thanks for the update. To be sure we're looking at the same thing, if we were to run the following code (based on what you posted), it would show the problem?

from picamera2 import Picamera2, Preview

picam2 = Picamera2()
camera_config = picam2.create_preview_configuration({"size":(400, 280)})
picam2.configure(camera_config)
picam2.set_controls({"ExposureTime":  10000, "Brightness": 0, "AeExposureMode": 1})

picam2.start_preview(Preview.QTGL, x=10, y=100, width=400, height=300)
picam2.title_fields = ["ExposureTime", "AnalogueGain"]

picam2.start()
array = picam2.capture_array()

Can you say how you are able to determine that the most recent frame is not displayed? I'm afraid it's not clear to me how you can tell.

You also mentioned an external trigger. Is the problem only apparent when using an external trigger? If so, how and when are you using the external trigger? Thanks.

[daodats]

@davidplowman, yes, the code would raise the problem. The issue appear when I use external trigger to active an array capture. The external trigger is pulled by an sensor that connect to the XTR pin of the Global Shutter Camera, the sensor pull the signal for each 2 seconds, so it is very easy to recognize this problem.

[davidplowman]

I'm just wondering if you're sending that pulse several times? There was a camera driver bug that was fixed quite recently whereby the delivered frame was always one late. So sending the 2nd pulse would give you the first frame, and so on. Your OS looks relatively old, would you be able to update it to see if that fixes the problem? As always, use a spare SD card or back up anything important first, just in case.

First try

sudo apt update
sudo apt full-upgrade

Actually I'm not 100% sure if the bug-fix is in the latest stable version, so if that doesn't make a difference it would be worth trying

sudo rpi-update

As I said, please ensure you've backed up anything critical first!

[daodats]

I have updated the OS and all related libs to the latest releases. Now, the preview window is able to show the most recent frame, but the capture_array still returned the old frame.

[davidplowman]

Thanks for the update. Could you just confirm what uname -a is now reporting?

Also, from what you say, I assume that when you run the short script that I posted, you're finding that array contains the image corresponding to the first pulse, but only when you send the second pulse. Is that correct?

Thanks.

[daodats]

Hi @davidplowman, the uname -a returned: Linux raspberrypi 6.6.25-v7+ #1751 SMP Fri Apr 5 15:15:08 BST 2024 armv7l GNU/Linux And, as you suggested. I had to send the second pulse to get the array that corresponding to the first pulse.

[davidplowman]

@njhollinghurst Would you know if the kernel version reported above would have the imx296 driver fix? Any other ideas on what might be happening here? Thanks!

[njhollinghurst]

The driver fix was here: https://github.com/raspberrypi/linux/pull/6010 merged on March 5, so I would expect it to be in that version.

It would explain why the preview window behaviour has changed on update. It does not explain why the captured image differs from the preview window, which sounds like not a driver problem.

[davidplowman]

The script above just captures the first image that it can, but obviously if there are more pulses the stream will continue, and those images should all get displayed. @daodats Can you say what is happening now if there is just one pulse - do you get any preview images and/or a captured frame at all? Perhaps we need to try and make that behave properly first.

[daodats]

Thanks @davidplowman and @njhollinghurst, If there is just one pulse, the preview windows doesn't show anything, and the preview only show image after several first pulses.

[davidplowman]

So I believe the we need to get the code to a state where just one pulse produces a single frame. @njhollinghurst Would you agree, and that anything else is basically wrong and needs fixing?

[njhollinghurst]

@daodats Could you check that Analogue gain and White Balance gains are all fixed before start? That should allow the first frame to be captured. If there are more pulses, the viewfinder may continue to update.

[daodats]

@njhollinghurst I had fixed Analogue gain value in this line of code: picam2.set_controls({"ExposureTime": g_exposure_time, "Brightness": 0, "AeExposureMode": 1} But, for the White Balance value, what is it corresponding parameter? is it "AwbEnable"?

[davidplowman]

Into your controls, add something like "ColourGains": (1.0, 1.0). If you want to find the correct values for the colour gains, you will need to run the sensor for a short while and query the metadata (picam2.capture_metadata()['ColourGains']).

[daodats]

Setting "ColourGains" still doesn't help. So I still workaround by sending the second pulse. Thank @davidplowman and @njhollinghurst !