ROI / Windowing do not work as expected at high framerate, how to solve ??

[Pe3ucTop]

Hello to all,

Need : ROI with resolution 640x480 with FPS 60...90 (ROI= Region of Interest= Camera Windowing). Problem: when defined ROI (in doc it's same Zoom) with resolution 640x80 and framerate 60 , it does not work as ROI (real 640x480) but work as zoom for ~320x240 region. Tested ok: When framerate defined as 10 , ROI work as expected . Posible explanation: When framerate is defined 60...90, driver select camera mode to 6 or 7 ignoring definition of ROI , and doing zoom of interested region.

Question: Is it possible to force ROI at camera modes 6 or 7 ? Can anybody help to solve this problem?

[6by9]

Please provide a coded example that demonstrates this. The firmware shouldn't be ignoring the ROI in any mode, but I can't tell from your description what values you are setting.

[Pe3ucTop]

Here it will be, difference in picture is terrible ! I'm awaiting same picture, because ROI must work in HW in sensor not at Image processor. Correct me if I'm wrong.

import time
import picamera

with picamera.PiCamera() as camera:
        camera.zoom = (0,0,.24,.24)
        camera.resolution = (640, 480)
        camera.framerate = 10
        time.sleep(.5)
        camera.start_preview()
        print("Priview at 10FPS mode..." )
        time.sleep(10)
        camera.stop_preview()
        camera.framerate = 60
        camera.start_preview()
        time.sleep(10)
        print("Priview at 60FPS mode..." )
        camera.stop_preview()
        print("Stop test.")

[6by9]

I've finally had a chance to run this. Totally expected behaviour. ROI/zoom is digital zoom always done in the image processing pipe on the SoC.

In your first case the sensor will be producing 1296x972, so clipping 24% will leave 311x233 pixels and that happens to be almost exactly what you're asking for. 1:1 image resizing is very reasonable.

In your second case the sensor will be producing 640x480, so clipping 24% will leave 153x115 pixels. You're asking for 640x480, so it is having to upscale those pixels at almost 4:1. It will look rubbish. Don't do something so silly.

[Pe3ucTop]

Thank you ! "ROI/zoom is digital zoom always done in the image processing pipe on the SoC." - part give me full answer. I in mistake believed that PiCamera is more advanced than it is... I still wonder, why such simple thing for sensor hardware (ROI=Windowing) is not implemented and we have to relay on image processing in SoC ? I do not understand your explanation of first case - "the sensor will be producing 1296x972" ?? Where do you get this ?? Why not other resolution (example 2592x1944) ?? And why not real 640x480 , I define it...

But more correct questions is :

• Is it possible to control sensor hardware parameters for advanced users somehow (mmal, ...)? Parameters could be: ROI / Binning mode / ...
• Whom to ask / who is responsible about Raspberry Camera driver ? Is it believable to await such improvements or it will be wasted time?

[6by9]

Modes - http://picamera.readthedocs.io/en/release-1.10/fov.html#camera-modes

Have you ever tried programming sensors and setting up on sensor cropping in a generic way? It is often far from trivial . If you crop on the sensor you also have to crop lens shading tables and other tuning parameters. Seeing as exposure time is typically set as a function of line timing, you've also got to recompute the correct values for that and notify the rest of the system. Looking at the range of V4L2 sensor drivers available, those appear not to offer on sensor cropping either, probably for the reasons listed above.

As it happens I'm the one who supports the GPU camera firmware. No, I won't be looking to additional controls over on-sensor cropping or binning controls. You already have the full list of modes supported, details of the algorithm used for the automatic selection between them, and also a way to override the mode selection.

[waveform80]

Closing as I think this one's been answered satisfactorily; do feel free to re-open if you've further queries about this!

As an addendum, the docs in the next release include a greatly expanded Camera Hardware section which goes into some detail regarding the camera's operation, and should help understanding the (also expanded) lower level mmalobj API which will include an introductory tour.