[HOW-TO] Camera Flash

[mgineer85]

Describe what it is that you want to accomplish Hey, I want to build a mobile camera.

After allowing per tuning file cranking up the range for AnalogueGain to 16 for the imx708 the shutter speed got on the faster side. It helps to achieve sharp images if hand held.

But if it is getting darker or indoors, a flash could be useful to get decent lit sharp captures.

I think about a LED or a speedlite to add to the camera.

• For speedlite, it might be hard to get the right timing and it's unclear to me how to deal with the AGC algorithms when there is just a speedlite exposing one frame instead of a more permanent light.
• For LED the light usually cannot be as bright as with the speedlite, but I think it would be OK to tinker around with it in the beginning.

My questions:

• Which is the current status in libcamera/picamera2 to use a flash? Are there any examples/best practice since it is not part of libcamera itself (yet)? Best I could find so far is maybe use this implementation to control the actual flash? https://forums.raspberrypi.com/viewtopic.php?t=368437. Adding dtparam=cam1_sync=24 to config.txt seems to have no effect on RPI5.
• Specifically how to implement something like light metering for example based on 1 test frame illuminated vs 1 frame not illuminated to adjust the camera algorithms?

Additional context For reference, I found these related issues.

• https://github.com/raspberrypi/picamera2/issues/635
• https://github.com/raspberrypi/picamera2/issues/893#issuecomment-1855459186
• https://github.com/raspberrypi/picamera2/issues/891#issuecomment-1852330916
• https://forums.raspberrypi.com/viewtopic.php?t=190314
• https://forums.raspberrypi.com/viewtopic.php?p=2208717&hilit=flash#p2209016
• https://picamera.readthedocs.io/en/release-1.13/recipes2.html#using-a-flash-with-the-camera

[davidplowman]

There is no explicit support in libcamera for flash metering at the moment, so I think you would have to implement something for yourself. I think quite a bit of experimentation might be necessary. Here's what I might try and do.

1. Monitor the image exposure and gain, and the brightness of the image. You could get the brightness either from the statistics that the ISP produces (this would differ between Pi 5 and earlier Pis), or from a scaled down low resolution stream which you could analyse with numpy.

2. Fix the exposure and gain to the current values. Once that has taken effect, turn on the flash. Once you're sure you've got a frame illuminated by the flash (you could use capture_request with the flush parameter?), turn off the flash and analyse the brightness of the image.

3. Use these values to calculate the shutter/gain you want for your final capture. Be careful if you're using the low resolution stream because this image will have gamma applied to it, making it non-linear in terms of exposure vs. brightness.

4. For the capture, create a full resolution configuration and in it specify the exposure/gain controls that you want, all as part of that configuration. These will then be applied to the capture. Once the capture is done, set up a preview mode with the original exposure/gain in the configuration, and then, once that mode has re-started, set the AEC/AGC back to auto.

I'm not familiar with speedlite, but I'm guessing these are xenon flashes? This would pose quite a few challenges. Timing is very critical, in fact I've known cameras that expect you to use a xenon flash to have a hardware trigger line that gets pulsed at the right moment. Even worse, most of our cameras are rolling shutter cameras, so you can't even guarantee to illuminate the entire frame with a single pulse.

As regards the mechanics of getting an LED flash to come on, that's not really a Picamera2 question and I don't know the answer. Maybe try asking that on the Camera Board forum and you'll find someone who can help you. Good luck!

[mgineer85]

Hi, thank you for the elaborated answer. I was kind of expecting that I might need to find my own algorithm to apply. Any chance to rely on the internal libcamera AEC/AGC algorithms without need to reimplement them? If you have any idea about a suitable algorithm I could implement in python/numpy that would be great also.

I like to avoid switching between modes, because at the same time I want to synchronize two cameras as we found a way here: #1050

[davidplowman]

I don't think the existing AEC/AGC will help you as that's all very bound up with the runtime C++, the camera metadata and ISP statistics. I think re-inventing some of that in Python is going to be necessary.

First of all, using the low resolution image is complicated somewhat by the fact that gamma is applied to it, whereas ISP statistics are complicated by the fact that you have to unpack them, and they're different on a Pi 5 (don't know if that's an issue for you).

If you're using the low resolution image, it'll probably be YUV420 (it can be RGB on a Pi 5, but not earlier Pis). We probably only need the Y channel. I would also build yourself an inverse-gamma lookup table (you can find the gamma transform in the .json camera tuning file). Reducing the size of the low resolution image will leave you with less computation, though there will be a limit as to how low you can go. It might be 64x64, not sure.

So I'd do this:

1. Run your camera normally and push your low res Y image through your inverse-gamma table, then get the average. (I know gamma operates on RGB, not Y, but if you only have Y then this will have to do. Again, the alternative would be to use the ISP stats.)

2. Do the same for an image that has identical exposure/gain but where the flash is on. Again, push this through the inverse-gamma table and compute the average (or record the brightness from the ISP stats).

3. Now you can calculate what total exposure (exposure*gain) you want, based on a "target exposure value". This is likely to be quite low, say 40, for un-gamma-ed 8-bit numbers.

4. To calculate this value, bear in mind the following. The flash (if it has different levels) will scale the brightness delta between your non-flash and flash images. Both exposure and gain scale everything. (Xenon is a bit more complex in this respect - the effect of the flash scales with gain but not exposure.)

5. This will give you the total exposure that you want, but you'll still have to divide them up into shutter time and gain (what exposure profiles do in the built-in algorithm). If your flash has different levels then there's also a trade-off there. Mostly you probably want to go with flash at maximum brightness, unless that leads to an exposure so low you can't manage it (in which case, select the minimum achievable exposure and calculate the amount of flash that this leaves you with).

There's probably quite a lot to digest there - hope it helps!