Quick full image burst with different EVs for HDR

[BKLronin]

Hi!

Iḿ currently using the regular commaline version to capture these 3 exposures. However it is rather slow so using this library instead and cpaturing to 3 different numpy arrays and then saving tham all three should be faster.

For some reason I can´t set camera controls properly. This part is really confusing with the different methods and the still configurations beforehand.

I tried making 3 configurations for each and then tried to capture with the picam2.capture_arrays method but it seems this only for preview and main so different streams not different configurations.

The linear approach was:

picam2 = Picamera2()
config = picam2.create_still_configuration()
picam2.configure(config)
picam2.start()
time.sleep(1)
#picam2.set_controls({"AwbEnable": 1, "AeEnable": 1})

#picam2.set_controls({"AeConstraintMode": controls.AeConstraintModeEnum.Shadows})

picam2.set_controls({"ExposureValue": -1})
ev1 = picam2.capture_array()

picam2.set_controls({"ExposureValue": 0})
ev2 = picam2.capture_array()

picam2.set_controls({"ExposureValue": 1})
ev3 = picam2.capture_array()

print("Saving..")

image = Image.fromarray(ev1)
image.save(f"test_hdr0.jpg", quality=50)

image = Image.fromarray(ev2)
image.save(f"test_hdr1.jpg", quality=50)

image = Image.fromarray(ev3)
image.save(f"test_hdr2.jpg", quality=50)

[davidplowman]

Hi, this is indeed a tricky subject. There are a few things to be aware of.

1. The "ExposureValue" control does not increase the exposure instantly, it merely sets the AGC target higher (or lower). Thereafter it will take several frames to get there. You might do better to note the exposure time and gain the camera currently has and then to set larger (or smaller) values directly.

2. Even when you change the exposure time and gain directly, even this takes "a few frames" to happen. Currently there is no particularly good way to check for this other than by examining the metadata in the images you get back, but it is a live topic of discussion within the libcamera project (unfortunately with no resolution as yet).

[BKLronin]

Thank you very much, I can work with that I think.

So essentially I need to do something like.

• Take average screen measurement
• get picam2.camera_controls (With settings in it hopefully)
• Take first frame
• Alter precalculated shutter time for gain 1 and -1 EV
• Wait until picam2.camera_controls reflects the changes OR take sample frame and check for meta according to point 2
• Take second frame and so on....

EDIT: Looks like there is a capture_metadata function available that get these infos from the running caemraframes after picam start. This seems to be a good source for the apllied settings before cpaturing the array?

[davidplowman]

Yes, capture_metadata() is what you want to use. That will have the current camera exposure and gain settings. You can then double, halve (or whatever) these numbers and program those directly. Then you'll need to sit there and watch the metadata until you see the numbers you asked for.

As I said, there is a discussion going on here within the libcamera project, and we've made proposals for what we'd like to see happen, but it's still a work in progress.

[BKLronin]

That will be sufficient for my project. Currently the hardware HDR outperfroms my alghoritm and it is a lot faster. Mine takes like 10 minutes. So I might fallback to the hardware HDR or wherever the cam does this.

Picamera in general is super nice, I was essentially fascinated by the ease of ho you could directly contrl a camera with python. That has been a problem for many years and with the picamera 3 there is finally a really capable cam on the market to actually use it.

If the camera_controls just would return some kind of 'OK' like a cnc controller that would be awesome though.

[BKLronin]

For completion, this is how I implemented it: https://github.com/BKLronin/numpyHDR/blob/main/picamburst.py

[davidplowman]

Looks good! One thing to note is that you can capture the frames more quickly if you're prepared to send the next exposure time before you've received the previous one. But that is more complicated because you start to get the "what if the exposure time I asked for doesn't show up" problem - because the camera is quite hard realtime, but Linux isn't, you aren't completely guaranteed to get every single thing you ask for (though you are guaranteed to get the final thing that you send, if that makes sense).

[BKLronin]

I think I understood roughly what you mean. As it is a queue that get send or received I could track on my end when a I did a certain thing and pass it before I get the ok for the previous one. Pretty much a Gcode queue for cnc :P

I identifeid something else, I moved the capture point from the first array after the manual gain and ae lock. So its a bit faster now.