Kernel 4.19.65: set AWB greyworld mode via v4l2-ctl command

popoviciri commented 4 years ago

Following up on #1167 the new AWB greyworld mode cannot be set via v4l2-ctl command. In MotionEyeOS, this is the way AWB modes are set. Hopefully there is something obvious I am missing. To my understanding v4l2-ctl is just an interface for accessing the driver. Here is the output of the v4l2-ctl slightly stripped of irrelevant bits:

Driver Info:
        Driver name      : bm2835 mmal
        Card type        : mmal service 16.1
        Bus info         : platform:bcm2835-v4l2
        Driver version   : 4.19.65
        Capabilities     : 0x85200005
                Video Capture
                Video Overlay
                Read/Write
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x05200005
                Video Capture
                Video Overlay
                Read/Write
                Streaming
                Extended Pix Format

Camera Controls

                  auto_exposure 0x009a0901 (menu)   : min=0 max=3 default=0 value=0
         exposure_time_absolute 0x009a0902 (int)    : min=1 max=10000 step=1 default=1000 value=1
     exposure_dynamic_framerate 0x009a0903 (bool)   : default=0 value=0
             auto_exposure_bias 0x009a0913 (intmenu): min=0 max=24 default=12 value=12
      white_balance_auto_preset 0x009a0914 (menu)   : min=0 max=9 default=1 value=1
            image_stabilization 0x009a0916 (bool)   : default=0 value=0
                iso_sensitivity 0x009a0917 (intmenu): min=0 max=4 default=0 value=3
           iso_sensitivity_auto 0x009a0918 (menu)   : min=0 max=1 default=1 value=1
         exposure_metering_mode 0x009a0919 (menu)   : min=0 max=2 default=0 value=0
                     scene_mode 0x009a091a (menu)   : min=0 max=13 default=0 value=0

Actual behaviour Note the white_balance_auto_preset 0x009a0914 (menu) : min=0 max=9 default=1 value=1 line. There is no setting 10 achievable via the v4l2-ctl command.

Use with picamera Perhaps related, following the changes in the PR on the python-picamera library https://github.com/waveform80/picamera/pull/576, result in the following error when passing the greyworld mode to python-picamera:

  File "usr/lib/python2.7/site-packages/picamera/camera.py", line 3014, in _set_
picamera.exc.PiCameraValueError: Invalid auto-white-balance mode: `greyworld`

System

Which model of Raspberry Pi? multiple rpis: Pi2B, Pi3B, PiZeroW, PiZero
Which OS and version (cat /etc/rpi-issue)? using MotioneyeOS custom build

Which firmware version (vcgencmd version)?

Aug  9 2019 17:28:49
Copyright (c) 2012 Broadcom
version 0e6daa5106dd4164474616408e0dc24f997ffcf3 (clean) (release) (start_x)

Which kernel version (uname -a)?
```
Linux cam6_lan 4.19.65-v7 #1 SMP Mon Aug 12 03:41:54 CEST 2019 armv7l GNU/Linux
```
Help Need help with a patch for the mmal.py file in python-picamera and help with parsing greyworld mode via v4l2-ctl command. Or a nudge to the right direction. Thanks! Cheers!

6by9 commented 4 years ago

The V4L2 spec doesn't have an enum for greyworld. Either we have to deviate from the spec, or it would be easy to combine it into the night scene mode option. Merging it into the scene mode will have the downside that exposure mode and metering mode will also be dictated by the scene mode rather than user selection.

Picamera is an independent project - issues related to it should remain there.

popoviciri commented 4 years ago

Hi @6by9, thanks for your feedback. Made this work with a patches for motion and picamera. Not for your regular user but one can easily patch either of the two. I know these are independent projects, however changes upstream have repercussions and users loose functionality. For example both v4l2rtspstreamer and gst-rtsp-server require a v4l2 device as source and I cannot believe I am the only one with the NoIR camera on /dev/video0. Obviously, downgrading is not a permanent option. So is there something you can do on the bcm2835-v4l2 driver to allow either a 10th enum for greyworld or the night scene hack? Personally I don't mind deviating from specs, specially if they're old. Thank you very much... Cheers!

EDIT: to be fair, you can also pipe from raspivid to gst-rtsp-server but you cannot use their test-launch example.

popoviciri commented 4 years ago

Hi @6by9, I've been trying to add the greyworld enum to v4l2 spec in raspberrypi/linux without success. So far I added the new mode to:

drivers/staging/vc04_services/bcm2835-camera/controls.c
drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
include/uapi/linux/v4l2-controls.h

It compiles well under buildroot and v4l2-ctl --all shows max 10 in awb: white_balance_auto_preset 0x009a0914 (menu) : min=0 max=10 default=1 value=4; however when I run v4l2-ctl --verbose -c white_balance_auto_preset=10 I am getting:

VIDIOC_QUERYCAP: ok
VIDIOC_S_EXT_CTRLS: failed: Invalid argument
Error setting controls: Invalid argument

I am doing something fundamentally wrong? Or is really not as easy to add this mode to v4l2 in linux? To my understanding the additions are counted fine and new enum should be picked by mmal as any other. Would appreciate any tip! Cheers!

6by9 commented 4 years ago

Without seeing the changes you've actually made it's nearly impossible to say what you've missed ir where it is failing. I'm compiling up the changes I'm expecting to need and throw them at a branch when I get a chance.

popoviciri commented 4 years ago

Yes, I realized that my question was impossible to answer without making clear the changes, apologies @6by9. This is what I did:

diff --git a/drivers/staging/vc04_services/bcm2835-camera/controls.c b/drivers/staging/vc04_services/bcm2835-camera/controls.c
index e49897b95..0641fff35 100644
--- a/drivers/staging/vc04_services/bcm2835-camera/controls.c
+++ b/drivers/staging/vc04_services/bcm2835-camera/controls.c
@@ -481,6 +481,10 @@ static int ctrl_set_awb_mode(struct bm2835_mmal_dev *dev,
        case V4L2_WHITE_BALANCE_SHADE:
                u32_value = MMAL_PARAM_AWBMODE_SHADE;
                break;
+
+       case V4L2_WHITE_BALANCE_GREYWORLD:
+               u32_value = MMAL_PARAM_AWBMODE_GREYWORLD;
+               break;
        }

        return vchiq_mmal_port_parameter_set(dev->instance, control,
@@ -1008,7 +1012,7 @@ static const struct bm2835_mmal_v4l2_ctrl v4l2_ctrls[V4L2_CTRL_COUNT] = {
        {
                V4L2_CID_AUTO_N_PRESET_WHITE_BALANCE,
                MMAL_CONTROL_TYPE_STD_MENU,
-               ~0x3ff, V4L2_WHITE_BALANCE_SHADE, V4L2_WHITE_BALANCE_AUTO, 0,
+               ~0x3ff, V4L2_WHITE_BALANCE_GREYWORLD, V4L2_WHITE_BALANCE_AUTO, 0:...skipping...
diff --git a/drivers/staging/vc04_services/bcm2835-camera/controls.c b/drivers/staging/vc04_services/bcm2835-camera/controls.c
index e49897b95..0641fff35 100644
--- a/drivers/staging/vc04_services/bcm2835-camera/controls.c
+++ b/drivers/staging/vc04_services/bcm2835-camera/controls.c
@@ -481,6 +481,10 @@ static int ctrl_set_awb_mode(struct bm2835_mmal_dev *dev,
        case V4L2_WHITE_BALANCE_SHADE:
                u32_value = MMAL_PARAM_AWBMODE_SHADE;
                break;
+
+       case V4L2_WHITE_BALANCE_GREYWORLD:
+               u32_value = MMAL_PARAM_AWBMODE_GREYWORLD;
+               break;
        }

        return vchiq_mmal_port_parameter_set(dev->instance, control,
@@ -1008,7 +1012,7 @@ static const struct bm2835_mmal_v4l2_ctrl v4l2_ctrls[V4L2_CTRL_COUNT] = {
        {
                V4L2_CID_AUTO_N_PRESET_WHITE_BALANCE,
                MMAL_CONTROL_TYPE_STD_MENU,
-               ~0x3ff, V4L2_WHITE_BALANCE_SHADE, V4L2_WHITE_BALANCE_AUTO, 0,
+               ~0x3ff, V4L2_WHITE_BALANCE_GREYWORLD, V4L2_WHITE_BALANCE_AUTO, 0,
                NULL,
                MMAL_PARAMETER_AWB_MODE,
                &ctrl_set_awb_mode,
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
index 416b4e1f0..38702d306 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
@@ -313,6 +313,7 @@ enum mmal_parameter_awbmode {
        MMAL_PARAM_AWBMODE_INCANDESCENT,
        MMAL_PARAM_AWBMODE_FLASH,
        MMAL_PARAM_AWBMODE_HORIZON,
+       MMAL_PARAM_AWBMODE_GREYWORLD,
 };

 enum mmal_parameter_imagefx {
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index e4ee10ee9..baa768253 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -815,6 +815,7 @@ enum v4l2_auto_n_preset_white_balance {
        V4L2_WHITE_BALANCE_FLASH                = 7,
        V4L2_WHITE_BALANCE_CLOUDY               = 8,
        V4L2_WHITE_BALANCE_SHADE                = 9,
+       V4L2_WHITE_BALANCE_GREYWORLD            = 10,
 };

 #define V4L2_CID_WIDE_DYNAMIC_RANGE            (V4L2_CID_CAMERA_CLASS_BASE+21)

I also realized that buildroot is pulling the compiled kernel from the firmware branch, albeit before the modules are copied from linux. I will keep trying and report back here if I have any success. Thanks!

6by9 commented 4 years ago

Only very brief testing, but the two patches on https://github.com/6by9/linux/tree/rpi-4.19.y-1 seem to do what is needed.

popoviciri commented 4 years ago

Hi again @6by9. I gave your branch a go and works as intended. Compiled in buildroot (thingos/motioneyeos) and can now set greyworld mode via v4l2-ctl enum 10:

v4l2-ctl --verbose -c white_balance_auto_preset=10
VIDIOC_QUERYCAP: ok
VIDIOC_S_EXT_CTRLS: ok

I can confirm this works on a rpi zero w and 3B so I can finally use my 6 NoIRs with v4l2rtspserver! What I missed was the 0x7FF over previous 0x3FF in menu declaration of the 10th mode in bcm2835-camera/controls.c. Honestly I don't know why this has to increase, but I guess that's the difference between devs and users. Thank you very much; I really appreciate you working it out!! This ends a struggle of weeks!

pelwell commented 4 years ago

Hint: 0x400 is (1 << 10).

popoviciri commented 4 years ago

And I though all programs are a combination of if/then's. I still don't get the shifts @pelwell but don't spoil it! Will get there for sure, thanks for the hint! And @6by9 thank you once again for the fix! Cheers!

raspberrypi / firmware

Kernel 4.19.65: set AWB greyworld mode via v4l2-ctl command #1225