raspberrypi / picamera2

New libcamera based python library
BSD 2-Clause "Simplified" License
901 stars 190 forks source link

Problems changing the FrameRate of Picam v3 using picamera2() #882

Open Carl0sC0elh0 opened 1 year ago

Carl0sC0elh0 commented 1 year ago

Hello everyone,

I'm trying to capture images to do real-time object counting. Everything works except for a small thing, the ROI is illuminated by a lamp with a small refresh rate so when the camera takes a shot there is all that dark noise from the lamp. I'm trying to reduce the refresh rate of the camera so that i can take care of that problem but i can't seem to do it.

This is a sample of the code, please have in mind im trying to work it out so it may have some unnecessary lines:

from picamera2 import Picamera2 from libcamera import controls import time import os

Cam = Picamera2()

os.system("v4l2-ctl --set-ctrl wide_dynamic_range=1 -d /dev/v4l-subdev0") Cam.still_configuration.size=(640, 480)

camera_config = Cam.create_still_configuration({"size": (640, 480)})

Cam.configure() Cam.set_controls({"FrameRate": 15}) Cam.set_controls({"AfMode": controls.AfModeEnum.Continuous}) Cam.start(show_preview = True)

size = Cam.capture_metadata()['ScalerCrop'][2:] full_res = Cam.camera_properties['PixelArraySize']

size = [int(s * 1.5) for s in size] offset = [(r - s) // 2 for r, s in zip(full_res, size)] Cam.set_controls({"ScalerCrop": offset + size}) input('Enter para tirar fotos')

Cam.start_and_capture_files("Foto{0:03}.jpg", initial_delay = 3, capture_mode = "still", num_files = 20, delay = 2)

os.system("v4l2-ctl --set-ctrl wide_dynamic_range=0 -d /dev/v4l-subdev0")

Anyone with a solution for this? Thanks in advance

davidplowman commented 1 year ago

Hi, thanks for the question. I think maybe we should start with a cut down example so that we can concentrate on the configuration and see what it does.

import time
import libcamera
from picamera2 import Picamera2, Preview

cam = Picamera2()
main = {'size': (640, 360)}
raw = {'size': (2304, 1296)}
preview_controls = {'FrameRate': 15}
preview_config = cam.create_preview_configuration(main, raw=raw, controls=preview_controls)
capture_controls = {'FrameRate': (2, 20)}
capture_config = cam.create_still_configuration(controls=capture_controls)
cam.configure(preview_config)
cam.set_controls({'AeExposureMode': libcamera.controls.AeExposureModeEnum.Long})
cam.start_preview(Preview.QTGL, width=640, height=360)
cam.start()

time.sleep(1)
cam.switch_mode_and_capture_file(capture_config, "test.jpg")

There are a few differences here:

Maybe see how these pictures look. Check with something like exiftool (sudo apt install exiftool if you don't have it) what the shutter time and ISO of the saved image is.

Carl0sC0elh0 commented 1 year ago

Thanks for the quick response, I added a input() line to check if the preview was indeed running at 15 and it is. Unfortunatly, i cant test the lighting right now but while we're at it, i want to save the image with a smaller resolution and the only way i got it don was by using Cam.still_configuration.size=(640, 480) now even if i use it with the code you provided, it doesn't work anymore.

Thank you again for your help.

davidplowman commented 1 year ago

Hi, if you're still running my example, try changing

capture_config = cam.create_still_configuration(controls=capture_controls)

to

capture_main = {'size': (640, 480)}
capture_raw = {'size': (4608, 2592)}
capture_config = cam.create_still_configuration(capture_main, raw=capture_raw, controls=capture_controls)

(And if you want VGA resolution probably change (640, 360) to (640, 480) everywhere!!) This is still using the full sensor resolution (4608, 2592) for the capture so will give the absolute best quality, but for such a small output resolution you might find that the preview mode (2304, 1296) is plenty.

I forget to mention that I'd removed the v4l2-ctl --set-ctrl ... thing. I'd leave that off for now as it somewhat confuses things by changing the available sensor modes. I'd try to get the basic script working and then one can always experiment.

Carl0sC0elh0 commented 1 year ago

Amazing! I can now test which resolutions will work best for my implementation. I'll try to update on the dark noise issue, don't close the issue just yet. Thank you @davidplowman !

Carl0sC0elh0 commented 1 year ago

Hello @davidplowman,

It works! The dark noise is no longer there. One last thing, can i use the example in zoom.py to apply in this implementation?

davidplowman commented 1 year ago

Great! Sure, those examples are all there for folks to use. If it doesn't work in any way, please feel free to file a new issue. Remember that a (really!) small test case that I can run myself is always helpful!

Carl0sC0elh0 commented 11 months ago

Hello again! I've been having problems with flickering and while searching the picam2 manual i found a control named 'AeFlickerPeriod'. I figure it is what I'm looking for but it does not work... I simply write somthing like

cam.set_controls({'AeFlickerPeriod': 10000})

and gives the output:

Traceback (most recent call last): File "/home/Carlos/Fischer/821.X/Testes/TiraPic.py", line 20, in cam.set_controls({'AeFlickerPeriod': 10000}) File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 1093, in set_controls self.controls.set_controls(controls) File "/usr/lib/python3/dist-packages/picamera2/controls.py", line 58, in set_controls self.setattr(k, v) File "/usr/lib/python3/dist-packages/picamera2/controls.py", line 33, in setattr raise RuntimeError(f"Control {name} is not advertised by libcamera") RuntimeError: Control AeFlickerPeriod is not advertised by libcamera

Is it not supported by picam v3?

davidplowman commented 11 months ago

It certainly should be available, and it seems OK for other folks here in the office. Could you confirm that all your software is up to date? Are you building anything for yourself? Thanks.

Carl0sC0elh0 commented 11 months ago

what software? If you mean the picamera2 library i certainlly did not update...I'm just trying to take some pictures and they get all this noise that i thought i handled earlier with reducing the frame rate... Could it be any problem relataed to libcamera not being updated?

Pls excuse my lack of knowledge on this matter

davidplowman commented 11 months ago

The normal procedure would be to do sudo apt update and sudo apt upgrade from time to time so that you pick up the latest version of libcamera, and a compatible version of Picamera2. The AeFlickerPeriod control was added, I think, many months ago.

Carl0sC0elh0 commented 11 months ago

That was the first thing i tried... I don'tknow if i have libcamera correclty installed, should i start there?

davidplowman commented 11 months ago

sudo apt update and sudo apt upgrade would update everything, and you'd be guaranteed to have up to date camera software. What does apt list python3-picamera2 report currently?

(Usual warnings about doing a full system update... if there's anything that you absolutely cannot afford to lose, then back it up somewhere else first!)

Carl0sC0elh0 commented 11 months ago

This is the output of apt list python3-picamera2:

python3-picamera/oldstable 1.13 arm64 python3-picamera/oldstable 1.13 armhf

davidplowman commented 11 months ago

Can you double-check that? The results I get all start with "python3-picamera2", not "python3-picamera". What version of the OS are you using (cat /etc/os-release) are you using?

Carl0sC0elh0 commented 11 months ago

Sorry I'm a noob, this is the output python3-picamera2/oldstable,oldstable,now 0.3.12-2 all [installed]

and the os is: PRETTY_NAME="Debian GNU/Linux 11 (bullseye)" NAME="Debian GNU/Linux" VERSION_ID="11" VERSION="11 (bullseye)" VERSION_CODENAME=bullseye ID=debian

davidplowman commented 11 months ago

OK, a little bit of digging here suggests that the last version of libcamera that's available with Bullseye did not have those flicker controls (Bullseye has become the "legacy OS" since the release of Bookworm).

Are you able to switch over to a Bookworm image?

Carl0sC0elh0 commented 11 months ago

I got a spare card but is that the only way?I remember picamera2 having some problems while working together with opencv. Nevertheless i'll see what i can do and I'll keep you posted.

davidplowman commented 11 months ago

Yes, it is recommended to do a clean installation rather than try to upgrade the entire OS in place. I don't believe there are any issues using OpenCV with Picamera2, it works on Bookworm like it always has - I'm sure you're familiar with the examples folder here.

Carl0sC0elh0 commented 11 months ago

Hello again! I switched to a bookworm image and tried to my script with

cam.set_controls({'AeFlickerPeriod': 10000}) cam.set_controls({'AeFlickerMode': libcamera.controls.AeFlickerModeEnum.Manual})

and nothing solved, no results with 'AeFlickerPeriod': 8333 either

tried to enable anti-flickering automatically with cam.set_controls({'AeFlickerMode': libcamera.controls.AeFlickerModeEnum.Auto}) and it doesnt work as well, saying in the output

[0:08:46.194088246] [2272] ERROR IPARPI ipa_base.cpp:872 Flicker mode 2 is not supported

Is there something i can do except turning off the lights?

davidplowman commented 11 months ago

Yes, there's no auto-detection supported currently. However, your first attempt should work. Try it again and see what

cam.capture_metadata()['ExposureTime']

reports.

Carl0sC0elh0 commented 11 months ago

It reports 681 if i dont configre it. However i tried changing it so small values using cam.set_controls({'ExposureTime': 200}) to reduce flickering that way and it doesn't work too...

davidplowman commented 11 months ago

The trouble is that for flicker cancellation to work, you have to have to be using an exposure time that is an integer multiple of the lighting period. So it can't help if the exposure time is less than 10ms.

If that's reporting 681, then that's 681us which is really low. Is the scene really that bright? There's no way flicker avoidance can help you here. I also find that modern lighting often causes much worse flicker than old incandescent bulbs (and is not always 100/120Hz either).

Carl0sC0elh0 commented 11 months ago

I'm afraid i can't change the lights, is there something i can do? Thanks for all the help!

davidplowman commented 11 months ago

Is it important that the exposure time is really short? If not (and this is a serious suggestion!) you could buy an ND filter to push up the exposure times.

Carl0sC0elh0 commented 11 months ago

It's not that i need small exposure times, i just want to eliminate the flickering :). I was trying to use exposure time to try and match the refresh rate of the lamps. Thank you for that insight!

I'll try and see if i can get info on those lamps.

Carl0sC0elh0 commented 11 months ago

The lamps are 100W LEDs, for curiosity, brightness enhances the flickering effect right? If the leds were weaker we can say the flickering would go away...?

davidplowman commented 11 months ago

The thing is that every pixel in the image "samples" the lighting for the exposure time. If all pixels were sampled at the same moment, then the entire image could be bright (the LEDs are "on"), or black (the LEDs are "off"). But because these are rolling shutter sensors, each pixel samples the light at a slightly later moment than the previous one, which gives you the banding effect.

LEDs are worse than incandescent bulbs because they go more quickly to "completely on" and more quickly to "completely off". The incandescent bulb takes a moment to heat up or cool down as the voltage changes, so there's some thermal inertia (there's probably a technical name for it) there.

The brightness as such doesn't change the flicker, but the exposure time does, because that determines how long you sample the light. If you can capture exactly one "lighting period", then every pixel gets the same amount of light. If you capture 1.5 lighting periods, you will still get flicker, but it will be less than, say, 0.5 lighting periods. Does this all make sense?

So lower power LEDs would help, doing effectively the same thing as putting an ND filter in front of the camera. But it only helps by causing longer exposures.

Carl0sC0elh0 commented 11 months ago

I'm getting close!! I think i've clearly matched the frequency of the lights but there is a very slow black band that passes through the whole image... have you heard of it?

davidplowman commented 11 months ago

That's exactly the kind of thing that lighting flicker causes. There's a period for which your light emits, well, no light, so the part of the image being exposed during that time ends up black. But because your frame period is not synced to the lighting, it ends up in a slightly different place in the image next time round. It ought to be possible to get the bands to be stationary if you can sync exactly, but you probably can't remove it completely without longer exposure times.