search

waveform80 / picamera

A pure Python interface to the Raspberry Pi camera module
https://picamera.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
1.57k stars 356 forks source link

Best settings for low light? #323

Closed chconnor closed 8 years ago

chconnor commented 8 years ago

Hi again!

I have a "Noir" security camera and am hoping to maximize is light sensitivity. Currently it's not gaining high enough in auto mode to see with my IR lighting. I'm wondering what the best settings are to maximize light gain... I'm happy to sacrifice performance in all other areas; e.g. noise is fine.

A few questions if someone has a sec:

Thanks!

waveform80 commented 8 years ago
  • what's the difference between "night" and "nightpreview" exposure modes?

No idea I'm afraid; the exact meaning of the exposure modes is a bit mysterious. At the moment "off" I understand: it simply fixes the sensor gains at their current values. "auto", the default, permits them to float (as do all other modes to my knowledge) until the AGC algorithm in the firmware decides the scene is nether under nor over-exposed. I presume the other modes adjust the assumptions of that algorithm in some way, but I couldn't say how precisely.

  • the best I have been able to do is to set exposure_mode to "nightpreview", not set iso, and set exposure compensation to 25. If I set it to "night" mode it isn't as bright, and setting iso=800 seems to help in night mode but not nightpreview mode... not sure why that is?

As far as I know, there's basically two physical things under the control of the exposure algorithm: the sensor gains (analog and digital), and the shutter speed (which is really the line readout time given we're dealing with a rolling shutter). There's probably several other "post-processing" things under the control of the exposure-modes, but those aren't exposed (ahem) by the firmware so we can't see what they change.

Setting iso to 800 forces the analog sensor gain "high" (how high exactly seems to depend on other factors, but higher than the 1.0-2.0 it has a tendency to sit at). As a result, in daylight, with exposure "auto", the exposure algorithm compensates by dropping shutter speed so the overall scene looks more or less the same.

As I don't know what "night" and "nightpreview" do precisely I can't say how they're affected by the iso setting. I'd hazard a guess that the major difference between "night" and "nightpreview" is that the latter is a similar optimization but for video applications whereas "night" might target still shots.

  • the lower the fps, the better it seems to do; is that impression correct?

Yes. The fps limits the shutter speed, so if your framerate is 30fps the shutter speed cannot be longer than 33.3333ms. By using a slower framerate you give the exposure algorithm more "freedom", i.e. a greater range of shutter speeds that it can choose from, and in the dark longer shutter speeds mean a longer line read-out time, so more photons hitting the sensor before the line gets read out.

  • is camera.brightness just a post-capture digital gain?

Yup. In fact it's post-post-capture digital gain. There's two sensor gains you can query: analog_gain and digital_gain. As I understand it, analog gain is set on the sensor itself, digital gain is applied by something (sensor ISP or the GPU, I'm not clear exactly which) afterward. Either way, I think digital gain is applied pretty early in the pipeline. Brightness (contrast, saturation, etc.) is applied toward the end of the pipeline (probably so they don't affect the assumptions of things like AGC and AWB, anyway they're post-post-capture :).

  • any other tips to squeeze the most of out the camera's ability to see in the dark? I haven't tried setting shutter_speed manually; I assumed it would just do 1/fps when in the night modes?

Test that assumption. It's probably true, but worth checking on. Here's a gist of a little script I've used in the past to play around with the gains and shutter speed to see what happens in various scenarios. You might try adding some more exposure modes to it and seeing what the gains and shutter speed do.

  • Possibly-unrelated question: what does drc_strength do?

Dynamic range compression ... I know what that means in the context of audio, but I've no specific idea in the case of imaging (other than "something vaguely similar but with pixel gains"). My guess would be there's a portion of the imaging pipeline that performs DRC on the pixel gains (early in the pipeline? later? I've no idea) and this adjusts how aggressive that function is. Put crudely, another form of contrast control.

Anyway - sorry I can't shed more light on this stuff - if the firmware devs contradict any of the above: listen to them, not me!

chconnor commented 8 years ago

Thanks! I'll continue tweaking.

6by9 commented 8 years ago

Ramblings from a firmware dev :-)

All the exposure modes set up different parameters within which the AE/AGC algorithm will work. eg

In all cases the selected frame rate will limit the maximum exposure time that can be achieved. You could select 'night' and 90fps, but it'd be nearly pointless. At the MMAL level there is the special frame rate value of '0', which means look at the MMAL_PARAMETER_FPS_RANGE values to set the max and min frame rates that you are happy for it to operate over. I haven't checked if that is exposed through PiCamera. Frame rate is also used as one of the main parameters in choosing which sensor mode to use, so do read the supported modes list and choose appropriately. Frame rate control works by altering the number of padding lines added to the end of the frame. They are never transmitted, but conceptually if the 1080P mode takes 33ms to read out the frame, then to get 15fps the sensor produces 2160 lines of data and the I2C commands to the sensor will say 2160 lines. (The V2.1 sensor does have some funky code in there to adjust the line length to get the very long exposure times, but we'll ignore that for now).

ISO modes are actually just another set of exposure modes, so setting the ISO makes the exposure mode setting irrelevant. For stills captures it will lock the combined analogue and digital gain at the requested value. During preview it is allowed to float a little, mainly so that flicker avoidance can choose the exposure time it needs whilst still achieving the correct overall settings. On V1.3 sensors (OV5647) ISO 100 = overall gain of x1.0. ISO200 = overall gain of x2.0, etc. On V2.1 sensors (IMX219) they have nominally calibrated it against the relevant standard (can't find a reference at the moment - jbeale has quoted it on the forums), so ISO100 = gain of x1.84. An overall gain of x1.0 will therefore report about ISO60, and ISO800 will be an overall gain of x14.72. Analogue gain is always preferred over digital as it produces less noise - you can't magic extra data into the LSB if you apply x2 digital gain.

exposure compensation adjusts the target Y value that the AE/AGC algorithm is aiming for. It still doesn't allow it to exceed the values defined by the exposure mode, nor change the frame rate.

All controls except analogue gain and exposure time are post processing of one form or another. Those two affect the sensor directly, the rest all control one or more processing stages in the SoC. I can't remember the exact order of processing stages in the pipe (there are about 20!). There are a couple of places that digital gain can be applied, but I seem to recall that waveform80 is right that the digital gain control that AGC uses is fairly early and in the Bayer domain. Contrast, saturation, brightness, etc, are all pretty late on in the YUV or YCbCr domain.

DRC goes hand in hand with High Dynamic Range (HDR) processing, but can be used in isolation. With HDR you've combined multiple frames to give extra definition to the light and dark areas, but you've got to encode this back to a standard 8bpp YUV frame for your JPEG. DRC is a non-linear squish of the histogram so that the detail in those dark and light areas is actually visible. It runs partly in the hardware pipe in some of the colour transforms, and partly as a software stage on the final frame, so it will have an impact on performance (I can't remember if it is on still captures only as well).

chconnor commented 8 years ago

Thanks for the very informative reply! That all dovetails perfectly with what I found while playing around, with a couple notes:

The camera is now shooting at 15fps, "nightpreview", exposure comp 25, and it's getting a very reasonable image from my IR lighting. As a bonus, the daytime image is also usable, despite those extreme settings: perhaps it is lowering the shutter speed to compensate during the day. I may still implement something in code to switch between "day" and "night" modes, but it's nice to have the option to not worry about it.

All the above info is very lucid and helpful; maybe a copy/paste into section 6 of the PiCamera docs?

Thanks again!

waveform80 commented 8 years ago

All the above info is very lucid and helpful; maybe a copy/paste into section 6 of the PiCamera docs?

Yup, there's some good stuff here - I'm going to merge it into the docs when I get a chance, but it'll need a little more nuance than a straight lift. There's some stuff in here I ought to add into the exposure_mode and iso properties for example, a fair bit for the camera hardware chapter, and some of the recipes will need a bit of re-working (I think there's a couple that set both exposure mode and iso which is obviously mis-leading in light of the above).