search

raspberrypi / linux

Kernel source tree for Raspberry Pi-provided kernel builds. Issues unrelated to the linux kernel should be posted on the community forum at https://forums.raspberrypi.com/
Other
11.02k stars 4.95k forks source link

Some V4L2 settings are missing/broken compared to the available MMAL settings #3988

Open dgsiegel opened 3 years ago

dgsiegel commented 3 years ago

Describe the bug

I would expect that all raspistill (which uses MMAL directly) would work for the V4L2 interface as well: https://www.raspberrypi.org/documentation/raspbian/applications/camera.md

To reproduce Compare https://www.raspberrypi.org/documentation/raspbian/applications/camera.md with v4l2-ctl -L

Expected behaviour Ideally all MMAL settings would be exposed to V4L2 as well

Actual behaviour

Compared to MMAL/raspistill/raspivid, v4l2-ctl is missing the following settings:

Exposure mode (scene_mode)

Only 3 values available:

Whereas raspistill offers:

Color effects (color_effects)

The following effects don't work:

The following effects are missing:

Colour effect (color_effects_cbcr)

Does not seem to have any effect

JPEG Quality (compression_quality)

Does not have any effect on MJPEG stream

Video stabilisation (image_stabilization)

Does not seem to have any effect

Metering (exposure_metering_mode)

Matrix metering is missing

Exposure mode (auto_exposure)

The following setting are missing:

Shutter speed (exposure_time_absolute)

Doesn't seem to have any effect above 800

Dynamic range compression

Does not exist in V4L2

Analog gain

Does not exist in V4L2

Digital gain

Does not exist in V4L2

Sensor mode

Does not exist in V4L2. Could be interesting for higher resolutions than 1080p with 2x2 binning

System

Affects all systems with 5.4.73 or later and the Raspberry HQ Camera.

The following kernel modules are affected:

6by9 commented 3 years ago

In a large number of those cases "Does not exist in V4L2" is exactly right.

V4L2 is a standard Linux API with documentation at https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/v4l2.html. Many of the controls from raspistill do not exist within the V4L2 API.

Whilst it is possible to add custom controls, doing so is a) platform specific b) difficult to do in a stable way

All the CID (Control ID) values are defined in https://elixir.bootlin.com/linux/latest/source/include/uapi/linux/v4l2-controls.h#L56. Add any downstream and there's a high chance that something else adds controls upstream and now shares your CID values.

Note:

dgsiegel commented 3 years ago

Thanks, this makes sense! What would be the best way however to expose all MMAL settings to the V4L2 API?

It would be very helpful for https://github.com/showmewebcam/showmewebcam which transforms the Pi into a UVC webcam.

lategoodbye commented 3 years ago

At least matrix metering should be available since Linux 5.8: https://github.com/raspberrypi/linux/commit/93a8883f5e4b96997f7221aca66cb45fbff9b379#diff-d501fb5d054c24c31bf3a2ee9869f60877469d9ba611599eab3580d04af26419

lategoodbye commented 3 years ago

Here is a patch against Linux 5.10 to add more scene modes (not sure about the right settings): https://gist.github.com/lategoodbye/f10e4cce20406bea901b168f244be916

Malvineous commented 2 years ago

I recently opened #4715 to add the sensor mode as a V4L2 custom control, but further discussion is needed so I'm going to reuse this issue as it covers the same ground.

I am quite keen to at least get support for changing the sensor mode implemented in V4L2, because using the Pi as a security camera this gives a much sharper image and is worth it despite the lower framerate.

In the PR, @6by9 was not keen on using V4L2 custom controls, so I would like to discuss what alternatives are available. For example I could add a new module option to set the value, or co-opt an unrelated V4L2 standard control instead. Neither of these seem that great to me, but I'm not entirely clear on the drawback of V4L2 custom controls.

6by9 pointed me to #4440, and according to that it is somewhat of a problem to keep BCM2835_ISP_BASE in sync between the Pi fork of the kernel and the upstream one. However my understanding is that this is already the situation, and this value now needs to be kept in sync whether we add more V4L2 custom controls or not, assuming we base them off the same constant. So if I am understanding correctly, it won't introduce any additional maintenance effort beyond what is already there.

Am I misunderstanding this? Is there more beyond this single definition that needs to be kept in sync? I am a little confused as the discussion in that issue makes it sound like libcamera is shipping its own version of v4l2-controls.h instead of using the version that matches whatever kernel is running on the system (typically located in /usr/include/linux/v4l2-controls.h and copied here from the kernel source code as part of the kernel installation process). If this is the case, libcamera will always risk using the wrong constant if it's built against a different v4l2-controls.h than the kernel was. I must be misunderstanding something, because if libcamera is using /usr/include/linux/v4l2-controls.h then it will always pick up the correct constant so #4440 wouldn't be an issue.

I am also wondering why this value needs to be kept in sync between the Pi fork and upstream. Is it possible to submit a preliminary upstream PR to allocate this constant permanently, and then just use the assigned value from that point on? Wouldn't this remove the hassle of having to always keep updating the value in the Pi fork every time upstream gets a new custom control allocation?

Sorry for all the questions, I'm just trying to understand the issues in the hope that there is some way to work around them!