Recording much darker than preview

[spikedrba]

Hello Silvan,

thanks a lot for all the great work again. I'm finding myself a bit confused by what I'm seeing when recording at night with a PiNoir and latest raspimjpeg/web interface and all default values.

Per default I'm using a divider of 1, which as far as I'm understanding means that preview and video recording use the same framerate. However as soon as I hit the record button the preview goes really dark and the result video is dark. As soon as I stop it the preview becomes bright again.

I brought this up on the forum and people pointed me to exposure time and fps, but as far I can understand the fps are the same, so I don't see why the preview would be much brighter.

any thoughts on this?

Thanks for all your help in advance,

Spike

[roberttidey]

I don't fully understand this but this is one thing that I think is relevant. When setting night mode the internal fps is automatically reduced to about 2 fps which allows for the enhanced exposure. However, when the capture is started the fps as set in the resolutions panel seems to override this, so if that was at 25fps then night mode is effectively lost. I find that if I reduce the video fps (and say boxing fps to about 2 fps during night mode then the recordings are better.

[spikedrba]

@roberttidey mmmh, ok, that makes sense, however I'm not sure I agree on the 2pfs and night mode, altho that's one other thing I've been desperately try to find out: what does exactly night mode do? In the specs for the chipset (www.ovt.com/uploads/parts/OV5647.pdf) it talks about various settings, but couldn't really make out what that translated to and how much control I had on it anyway.

The reason I'm not agreeing is that looking at a test recording the image was far too fluid to be 2fps and also testing in console:

>>> c.exposure_mode
u'auto'
>>> c.framerate
Fraction(30, 1)
>>> c.exposure_mode = 'night'
>>> c.framerate
Fraction(30, 1)
>>>

that said it ought to be something to do with exposure time, gain etc, I just can't figure out exactly what. Any input would be much appreciated.

thanks,

Spike

[silvanmelchior]

If you are in a dark room and set the exposure mode to night, you'll see that there are only a few frames per second left, so it seems as the driver of the cam ignores the fps-setting as long as there is no capture.

If you need the preview to be exactly as the video, you could set the circular buffer to 1000, smaller values might give a problem as I noticed right now.

[spikedrba]

@silvanmelchior apologies for bugging you again, but this is very important to my project and I can't reproduce your claim about night mode.

I'm testing with raspivid using this command: raspivid -v -set -ex night -fps 30 -o test.h264

which gives me the following stable stats through the recording after initial stabilization: mmal: Exposure now 33120, analog gain 2048/256, digital gain 511/256 mmal: AWB R=277/256, B=336/256

as you can see the exposure speed is what you'd expect from 30 fps. If I remove the fps setting it gives me the same exact results (and with -v I can see in the beginning picking that setting as default anyway):

raspivid Camera App v1.3.12

Width 1920, Height 1080, filename test.h264
bitrate 17000000, framerate 30, time delay 5000
H264 Profile high

I've repeated all those tests with a lens cap on to simulate complete darkness and got the same exact numbers.

It's only when I force fps to say 15 that I see the exposure changing.

mmal: Exposure now 66461, analog gain 1280/256, digital gain 387/256
mmal: AWB R=43/256, B=42/256

which is incidentally twice as with 30fps just as you would expect.

I'm never seeing framerate dropping to a few per second when enabling night mode, in fact night mode seems to basically do nothing for me for some reason, or maybe my tests are bogus?

thanks,

[silvanmelchior]

This seems to be correct, because raspivid is recording during your tests and the camera driver seems to force the full framerate while recording. If you test it on the webinterface without buffer and not recording, enable night mode and go into a dark room, jou should see the low fps if you wave your hand in front of the camera for example.

Do you get the same result?

[roberttidey]

So, if I understand this correctly then the exposure mode settings like night are being applied to the data fed to the jpeg call backs when not in video capture mode and so it sets conditions / fps to what is required. When video recording is turned on either by buffering or recording then the data to both h264 callbacks and jpeg callbacks is effectively using the fps as set by video recording parameters and em (night) has no effect.

That sort of implies that em is really intended for still captures. It would have been nice if the driver just lowered video fps when these modes are selected but I guess having an 'unpredictable' fps would cause difficulties.

Although one can lower the video fps setting to get some extra exposure its not the same as night itself so there are other parameters being affected. What would be interesting is to see if one can simulate a closer video night mode with a combination of parameter changes.

[spikedrba]

ok now I'm beginning to understand.

@silvanmelchior so what you're saying is that the night mode has an effect on the preview, not the recording. Am I understanding you right?

if that's the case then I'm with @roberttidey being puzzled by the value of it, is it just for still captures?

But most important of all is @roberttidey's last point: what would be the closest to resembling night mode for video recording? right now I'm using the schedule to lower fps and increase brightneed and contrast, but I feel that's not quite it and there's more that could be done with gains and exposure, I just have no idea what.

[silvanmelchior]

Yes, I think this is correct, but I'm not sure of course. But it makes sense, in the night there is not enough light, so the sampling needs to be longer and this results in the low fps. But since the recording is set to a higher framerate, the night setting can't be achieved while capturing. But for still images or for videos with a low fps it ahoud be possible.

I don't know the best settings either, but I think the only way to find it out is trying.

[spikedrba]

I guess I was confused by the other mode, nightpreview. I would have expected that to do what you described and affect the preview while the other would have overridden fps and whatnot to adapt the video (maybe after you set fps to 0, there's several combo like shutter_speed/exposure_speed that are autodetermined by the camera if set to 0 and manual if set to anything else).

And actually I was just poking around and found this, somewhat old, thread there referenced this comment: https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=76909

Setting fps to 0 tells the camera to use dynamic frame rates. You will get 30 fps until your exposure time is too long to archive 30fps.

This could be an interesting mode for a security camera project, I'm gonna experiment with it and let it run for a day, altho I can't test with raspimjpeg/the webapp because I don't see a way to log this stuff out so will test with picamera/raspivid.

two other interesting things from that thread:

• it discusses how ISO is the only way to influence the gains, however no matter what ISO I set in the webinterface I see no changes in the image. This is another thing I don't understand
• the thread mentions that position of arguments matter, is if you set the shutter speed first and then exposure latter will overwrite former. I don't know if this kind of thing is affecting raspimjpeg in any way and what that means to ISO settings (and therefore the gains)

This is also relatively old but has some points especially about night mode, James was taking a closer look and things don't seem to have changed much since 2011... https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=65163&p=479465

I've taken a look at the tuning for night mode and it definitely doesn't take advantage of the new longer exposures. It also has a fairly low max gain (8x rather than 16x in some of the other modes). Might be able to get a bit more analogue and digital game in that mode to give better very low light performance (but noisy). I think this is indeed down to trying to keep handshake effect down to a minimum, but as you say, Raspi's don't normally go around handheld.

also from another more recent comment to that thread, same problem:

I've tried setting sh (0), co (0), br (50), sa (0), ISO (100), ev (0), ex (fixedfps), awb (off), awbg (1,1) to constant values, but the cam still automatically adjust itself to light intensity.

[spikedrba]

more interesting stuff. it seems that night mode after all does play a role in video mode as well as long as you don't force the fps. After finding out about the 0 fps I tried that out and observed the following behavior:

raspivid -v -set -vf -fps 0 -t 0 -o /dev/null (without night mode)

mmal: Exposure now 32972, analog gain 304/256, digital gain 928/256
mmal: AWB R=284/256, B=311/256
mmal: Exposure now 32972, analog gain 304/256, digital gain 928/256
mmal: AWB R=285/256, B=310/256
mmal: Exposure now 50959, analog gain 640/256, digital gain 282/256
mmal: AWB R=286/256, B=309/256
mmal: Exposure now 50959, analog gain 640/256, digital gain 321/256
mmal: AWB R=286/256, B=308/256
mmal: Exposure now 59716, analog gain 640/256, digital gain 290/256
mmal: AWB R=286/256, B=308/256
mmal: Exposure now 62999, analog gain 672/256, digital gain 261/256
mmal: AWB R=286/256, B=308/256

it basically goes up from the normal shutter speed for 30fps to the speed for 15fps (62ms). But check out what it prints out if I enable night mode:

mmal: Exposure now 115215, analog gain 384/256, digital gain 256/256
mmal: AWB R=286/256, B=309/256
mmal: Exposure now 115215, analog gain 384/256, digital gain 256/256
mmal: AWB R=287/256, B=308/256
mmal: Exposure now 117049, analog gain 384/256, digital gain 256/256
mmal: AWB R=287/256, B=308/256

which is ~8fps (still ok for motion detection btw and watching a stream, professional security installation work at 7.5fps for standard monitoring, our eye can't tell the difference >=7.5)

regarding iso settings and gains, I'm not quite sure how to interpret those analog and digital gain settings, I've read about 8x etc, but there it has a fraction with /256 which I don't know what it means, altho I could guess 8x as ratio of the fraction, but not sure. The one think I noticed tho is that without setting any ISO it went as high as 672 from the paste above, while with -ISO 800 it went up to 944. The other thing is that if I set a fixed fps (meaning it can't change shutter speed that much), gains will vary a lot more:

raspivid -v -set -vf -fps 30 -t 0 -o /dev/null

mmal: Exposure now 33120, analog gain 1968/256, digital gain 256/256
mmal: AWB R=273/256, B=329/256

digital gain never seems to change much tho. Testing this with 15 fps confirms the behavior, shutter speed doubled and the AG stayed around 640. The thing I don't quite understand if I set ISO to anything the AG will now fluctuate much more and go up to 900 (regardless if I set ISO 100 or 800).

sorry for the flood, I thought you might find this info useful, happy to stop posting if it's just too much [noise].

Spike

[spikedrba]

also there is an exposure mode "off" that should help locking settings

https://github.com/raspberrypi/userland/blob/daad93e972f2332555ca2dc7b5a27a89d52572d5/host_applications/linux/apps/raspicam/RaspiCamControl.h#L35

but it doesn't seem to be implemented in raspivid:

pi@raspberrypi ~ $ raspivid -v -set -vf -fps 15 -ex off -t 0 -o /dev/null
RaspiVid: Unknown exposure mode: off

raspivid Camera App v1.3.12

https://github.com/raspberrypi/userland/blob/master/host_applications/linux/apps/raspicam/RaspiCamControl.c#L45

EDIT: just found a PR for it with an explanation: https://github.com/raspberrypi/userland/pull/171