Add support for OV13850 DUAL camera

Hi @initBasti

I have added support for a dual camera that seems to be OK but I can not figure out how to configure the cameras.

Here is the device tree for the second camera:

rk3399.dtsi:

    isp1: isp1@ff920000 {
        compatible = "rockchip,rk3399-cif-isp";
        reg = <0x0 0xff920000 0x0 0x4000>;
        interrupts = <GIC_SPI 44 IRQ_TYPE_LEVEL_HIGH 0>;
        clocks = <&cru SCLK_ISP1>,
             <&cru ACLK_ISP1_WRAPPER>,
             <&cru HCLK_ISP1_WRAPPER>;
        clock-names = "isp", "aclk", "hclk";
        iommus = <&isp1_mmu>;
        phys = <&mipi_dphy_rx0>;
        phy-names = "dphy";
        power-domains = <&power RK3399_PD_ISP1>;
        status = "disabled";

        ports {
            #address-cells = <1>;
            #size-cells = <0>;

            port@0 {
                reg = <0>;
                #address-cells = <1>;
                #size-cells = <0>;
            };
        };
    };

*rk3399-nanopi-:**

&i2c2 {
    status = "okay";
    ov13850p1: ov13850@10 {
        compatible = "ovti,ov13850";
        status = "okay";
        reg = <0x10>;
        clocks = <&cru 0x89>;
        clock-names = "xvclk";
        avdd-supply = <&ov13850_avdd_2p8v>;
        dovdd-supply = <&ov13850_dovdd_1p8v>;
        dvdd-supply = <&ov13850_dvdd_1p2v>;

        reset-gpios = <&gpio0 8 GPIO_ACTIVE_HIGH>;
        pwdn-gpios = <&gpio0 12 GPIO_ACTIVE_HIGH>;
        pinctrl-names = "rockchip,camera_default", "rockchip,camera_sleep";
        pinctrl-0 = <&cam1_default_pins &cif_clkout_a>;
        pinctrl-1 = <&cam1_default_pins>;

        port {
            ucam_out1b: endpoint {
                remote-endpoint = <&mipi_in_ucam1b>;
                data-lanes = <1 2>;
            };
        };
    };
};

&isp1_mmu {
    status = "okay";
};

&isp1 {
    status = "okay";

    ports {
        port@0 {
            mipi_in_ucam1b: endpoint@0 {
                reg = <0>;
                remote-endpoint = <&ucam_out1b>;
                data-lanes = <1 2>;
            };
        };
    };
};

In theory, both OV13850 are loaded and available:

ubuntu@nanopi-m4:~$ cat /sys/dev/char/*/device/name
gpio-keys
SIGMACHIP USB Keyboard
SIGMACHIP USB Keyboard Consumer Control
SIGMACHIP USB Keyboard System Control
USB Optical Mouse
USB Optical Mouse
ov13850
ov13850
rk3x-i2c
rk3x-i2c
rk3x-i2c
rk3x-i2c
rk3x-i2c
ubuntu@nanopi-m4:~$

and

ubuntu@nanopi-m4:~$ ls -d /dev/video*
/dev/video0  /dev/video10  /dev/video2  /dev/video4  /dev/video6  /dev/video8
/dev/video1  /dev/video11  /dev/video3  /dev/video5  /dev/video7  /dev/video9

Do you have another camera? Anyway below is all information that I think is relevant, can you spot how it should be configured?

ubuntu@nanopi-m4:~$ /usr/local/bin/v4l2-ctl -d /dev/video0 --all
Driver Info:
    Driver name      : rockchip-rga
    Card type        : rockchip-rga
    Bus info         : platform:rga
    Driver version   : 5.11.0
    Capabilities     : 0x84208000
        Video Memory-to-Memory
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04208000
        Video Memory-to-Memory
        Streaming
        Extended Pix Format
Priority: 2
Format Video Capture:
    Width/Height      : 100/100
    Pixel Format      : 'BA24' (32-bit ARGB 8-8-8-8)
    Field             : None
    Bytes per Line    : 400
    Size Image        : 40000
    Colorspace        : Default
    Transfer Function : Default (maps to Rec. 709)
    YCbCr/HSV Encoding: Default (maps to ITU-R 601)
    Quantization      : Default (maps to Full Range)
    Flags             : 
Format Video Output:
    Width/Height      : 100/100
    Pixel Format      : 'BA24' (32-bit ARGB 8-8-8-8)
    Field             : None
    Bytes per Line    : 400
    Size Image        : 40000
    Colorspace        : Default
    Transfer Function : Default (maps to Rec. 709)
    YCbCr/HSV Encoding: Default (maps to ITU-R 601)
    Quantization      : Default (maps to Full Range)
    Flags             : 
Selection Video Capture: compose, Left 0, Top 0, Width 100, Height 100, Flags: 
Selection Video Capture: compose_default, Left 0, Top 0, Width 100, Height 100, Flags: 
Selection Video Capture: compose_bounds, Left 0, Top 0, Width 100, Height 100, Flags: 
Selection Video Output: crop, Left 0, Top 0, Width 100, Height 100, Flags: 
Selection Video Output: crop_default, Left 0, Top 0, Width 100, Height 100, Flags: 
Selection Video Output: crop_bounds, Left 0, Top 0, Width 100, Height 100, Flags: 

User Controls

                horizontal_flip 0x00980914 (bool)   : default=0 value=0
                  vertical_flip 0x00980915 (bool)   : default=0 value=0
                         rotate 0x00980922 (int)    : min=0 max=270 step=90 default=0 value=0 flags=modify-layout
               background_color 0x00980923 (int)    : min=0 max=16777215 step=1 default=0 value=0
ubuntu@nanopi-m4:~$ /usr/local/bin/v4l2-ctl -d /dev/video1 --all
Driver Info:
    Driver name      : rkvdec
    Card type        : rkvdec
    Bus info         : platform:rkvdec
    Driver version   : 5.11.0
    Capabilities     : 0x84204000
        Video Memory-to-Memory Multiplanar
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04204000
        Video Memory-to-Memory Multiplanar
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : rkvdec
    Model            : rkvdec
    Serial           : 
    Bus info         : platform:rkvdec
    Media version    : 5.11.0
    Hardware revision: 0x00000000 (0)
    Driver version   : 5.11.0
Interface Info:
    ID               : 0x0300000c
    Type             : V4L Video
Entity Info:
    ID               : 0x00000001 (1)
    Name             : rkvdec-source
    Function         : V4L2 I/O
    Pad 0x01000002   : 0: Source
      Link 0x02000008: to remote pad 0x1000004 of entity 'rkvdec-proc': Data, Enabled, Immutable
Priority: 2
Format Video Capture Multiplanar:
    Width/Height      : 48/48
    Pixel Format      : 'NV12' (Y/CbCr 4:2:0)
    Field             : None
    Number of planes  : 1
    Flags             : 
    Colorspace        : Rec. 709
    Transfer Function : Default
    YCbCr/HSV Encoding: Default
    Quantization      : Default
    Plane 0           :
       Bytes per Line : 48
       Size Image     : 4608
Format Video Output Multiplanar:
    Width/Height      : 48/48
    Pixel Format      : 'S264' (H.264 Parsed Slice Data)
    Field             : None
    Number of planes  : 1
    Flags             : 
    Colorspace        : Rec. 709
    Transfer Function : Default
    YCbCr/HSV Encoding: Default
    Quantization      : Default
    Plane 0           :
       Bytes per Line : 0
       Size Image     : 4608

Codec Controls

                     h264_level 0x00990a67 (menu)   : min=0 max=15 default=0 value=0 (1)
                0: 1
                1: 1b
                2: 1.1
                3: 1.2
                4: 1.3
                5: 2
                6: 2.1
                7: 2.2
                8: 3
                9: 3.1
                10: 3.2
                11: 4
                12: 4.1
                13: 4.2
                14: 5
                15: 5.1
                   h264_profile 0x00990a6b (menu)   : min=0 max=4 default=2 value=2 (Main)
                0: Baseline
                1: Constrained Baseline
                2: Main
                4: High

Stateless Codec Controls

               h264_decode_mode 0x00a40900 (menu)   : min=1 max=1 default=1 value=1 (Frame-Based)
                1: Frame-Based
                h264_start_code 0x00a40901 (menu)   : min=1 max=1 default=1 value=1 (Annex B Start Code)
                1: Annex B Start Code
    h264_sequence_parameter_set 0x00a40902 (unknown): type=200 value=unsupported payload type flags=has-payload
     h264_picture_parameter_set 0x00a40903 (unknown): type=201 value=unsupported payload type flags=has-payload
            h264_scaling_matrix 0x00a40904 (unknown): type=202 value=unsupported payload type flags=has-payload
         h264_decode_parameters 0x00a40907 (unknown): type=204 value=unsupported payload type flags=has-payload
ubuntu@nanopi-m4:~$ /usr/local/bin/v4l2-ctl -d /dev/video2 --all
Driver Info:
    Driver name      : rkisp1
    Card type        : rkisp1
    Bus info         : platform:rkisp1
    Driver version   : 5.11.0
    Capabilities     : 0xa4201000
        Video Capture Multiplanar
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x24201000
        Video Capture Multiplanar
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : rkisp1
    Model            : rkisp1
    Serial           : 
    Bus info         : platform:rkisp1
    Media version    : 5.11.0
    Hardware revision: 0x00000000 (0)
    Driver version   : 5.11.0
Interface Info:
    ID               : 0x0300000d
    Type             : V4L Video
Entity Info:
    ID               : 0x0000000c (12)
    Name             : rkisp1_mainpath
    Function         : V4L2 I/O
    Pad 0x0100000f   : 0: Sink
      Link 0x02000022: from remote pad 0x1000008 of entity 'rkisp1_resizer_mainpath': Data, Enabled, Immutable
Priority: 2
Video input : 0 (rkisp1_mainpath: ok)
Format Video Capture Multiplanar:
    Width/Height      : 1280/720
    Pixel Format      : 'YUYV' (YUYV 4:2:2)
    Field             : None
    Number of planes  : 1
    Flags             : 
    Colorspace        : Default
    Transfer Function : Default
    YCbCr/HSV Encoding: Default
    Quantization      : Default
    Plane 0           :
       Bytes per Line : 2560
       Size Image     : 1843200
ubuntu@nanopi-m4:~$ /usr/local/bin/v4l2-ctl -d /dev/video3 --all
Driver Info:
    Driver name      : rkisp1
    Card type        : rkisp1
    Bus info         : platform:rkisp1
    Driver version   : 5.11.0
    Capabilities     : 0xa4201000
        Video Capture Multiplanar
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x24201000
        Video Capture Multiplanar
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : rkisp1
    Model            : rkisp1
    Serial           : 
    Bus info         : platform:rkisp1
    Media version    : 5.11.0
    Hardware revision: 0x00000000 (0)
    Driver version   : 5.11.0
Interface Info:
    ID               : 0x03000011
    Type             : V4L Video
Entity Info:
    ID               : 0x00000010 (16)
    Name             : rkisp1_selfpath
    Function         : V4L2 I/O
    Pad 0x01000013   : 0: Sink
      Link 0x02000026: from remote pad 0x100000b of entity 'rkisp1_resizer_selfpath': Data, Enabled, Immutable
Priority: 2
Video input : 0 (rkisp1_selfpath: ok)
Format Video Capture Multiplanar:
    Width/Height      : 1280/720
    Pixel Format      : 'YUYV' (YUYV 4:2:2)
    Field             : None
    Number of planes  : 1
    Flags             : 
    Colorspace        : Default
    Transfer Function : Default
    YCbCr/HSV Encoding: Default
    Quantization      : Default
    Plane 0           :
       Bytes per Line : 2560
       Size Image     : 1843200
ubuntu@nanopi-m4:~$ /usr/local/bin/v4l2-ctl -d /dev/video4 --all
Driver Info:
    Driver name      : rkisp1
    Card type        : rkisp1_stats
    Bus info         : platform:rkisp1
    Driver version   : 5.11.0
    Capabilities     : 0x84a00000
        Metadata Capture
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04a00000
        Metadata Capture
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : rkisp1
    Model            : rkisp1
    Serial           : 
    Bus info         : platform:rkisp1
    Media version    : 5.11.0
    Hardware revision: 0x00000000 (0)
    Driver version   : 5.11.0
Interface Info:
    ID               : 0x03000016
    Type             : V4L Video
Entity Info:
    ID               : 0x00000014 (20)
    Name             : rkisp1_stats
    Function         : V4L2 I/O
    Pad 0x01000015   : 0: Sink
      Link 0x0200002a: from remote pad 0x1000005 of entity 'rkisp1_isp': Data, Enabled, Immutable
Priority: 2
Format Metadata Capture:
    Sample Format   : 'RK1S' (Rockchip ISP1 3A Statistics)
    Buffer Size     : 108
ubuntu@nanopi-m4:~$ /usr/local/bin/v4l2-ctl -d /dev/video5 --all
Driver Info:
    Driver name      : rkisp1
    Card type        : rkisp1_params
    Bus info         : platform:rkisp1
    Driver version   : 5.11.0
    Capabilities     : 0x8c200000
        Metadata Output
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x0c200000
        Metadata Output
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : rkisp1
    Model            : rkisp1
    Serial           : 
    Bus info         : platform:rkisp1
    Media version    : 5.11.0
    Hardware revision: 0x00000000 (0)
    Driver version   : 5.11.0
Interface Info:
    ID               : 0x0300001a
    Type             : V4L Video
Entity Info:
    ID               : 0x00000018 (24)
    Name             : rkisp1_params
    Function         : V4L2 I/O
    Pad 0x01000019   : 0: Source
      Link 0x02000028: to remote pad 0x1000003 of entity 'rkisp1_isp': Data, Enabled, Immutable
Priority: 2
Format Metadata Output:
    Sample Format   : 'RK1P' (Rockchip ISP1 3A Parameters)
    Buffer Size     : 2964
ubuntu@nanopi-m4:~$ /usr/local/bin/v4l2-ctl -d /dev/video6 --all
Driver Info:
    Driver name      : rkisp1
    Card type        : rkisp1
    Bus info         : platform:rkisp1
    Driver version   : 5.11.0
    Capabilities     : 0xa4201000
        Video Capture Multiplanar
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x24201000
        Video Capture Multiplanar
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : rkisp1
    Model            : rkisp1
    Serial           : 
    Bus info         : platform:rkisp1
    Media version    : 5.11.0
    Hardware revision: 0x00000000 (0)
    Driver version   : 5.11.0
Interface Info:
    ID               : 0x0300000d
    Type             : V4L Video
Entity Info:
    ID               : 0x0000000c (12)
    Name             : rkisp1_mainpath
    Function         : V4L2 I/O
    Pad 0x0100000f   : 0: Sink
      Link 0x02000022: from remote pad 0x1000008 of entity 'rkisp1_resizer_mainpath': Data, Enabled, Immutable
Priority: 2
Video input : 0 (rkisp1_mainpath: ok)
Format Video Capture Multiplanar:
    Width/Height      : 1920/1080
    Pixel Format      : 'YUYV' (YUYV 4:2:2)
    Field             : None
    Number of planes  : 1
    Flags             : 
    Colorspace        : Default
    Transfer Function : Default
    YCbCr/HSV Encoding: Default
    Quantization      : Default
    Plane 0           :
       Bytes per Line : 3840
       Size Image     : 4147200
ubuntu@nanopi-m4:~$ /usr/local/bin/v4l2-ctl -d /dev/video7 --all
Driver Info:
    Driver name      : rkisp1
    Card type        : rkisp1
    Bus info         : platform:rkisp1
    Driver version   : 5.11.0
    Capabilities     : 0xa4201000
        Video Capture Multiplanar
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x24201000
        Video Capture Multiplanar
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : rkisp1
    Model            : rkisp1
    Serial           : 
    Bus info         : platform:rkisp1
    Media version    : 5.11.0
    Hardware revision: 0x00000000 (0)
    Driver version   : 5.11.0
Interface Info:
    ID               : 0x03000011
    Type             : V4L Video
Entity Info:
    ID               : 0x00000010 (16)
    Name             : rkisp1_selfpath
    Function         : V4L2 I/O
    Pad 0x01000013   : 0: Sink
      Link 0x02000026: from remote pad 0x100000b of entity 'rkisp1_resizer_selfpath': Data, Enabled, Immutable
Priority: 2
Video input : 0 (rkisp1_selfpath: ok)
Format Video Capture Multiplanar:
    Width/Height      : 1280/720
    Pixel Format      : 'YUYV' (YUYV 4:2:2)
    Field             : None
    Number of planes  : 1
    Flags             : 
    Colorspace        : Default
    Transfer Function : Default
    YCbCr/HSV Encoding: Default
    Quantization      : Default
    Plane 0           :
       Bytes per Line : 2560
       Size Image     : 1843200
ubuntu@nanopi-m4:~$ /usr/local/bin/v4l2-ctl -d /dev/video8 --all
Driver Info:
    Driver name      : rkisp1
    Card type        : rkisp1_stats
    Bus info         : platform:rkisp1
    Driver version   : 5.11.0
    Capabilities     : 0x84a00000
        Metadata Capture
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04a00000
        Metadata Capture
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : rkisp1
    Model            : rkisp1
    Serial           : 
    Bus info         : platform:rkisp1
    Media version    : 5.11.0
    Hardware revision: 0x00000000 (0)
    Driver version   : 5.11.0
Interface Info:
    ID               : 0x03000016
    Type             : V4L Video
Entity Info:
    ID               : 0x00000014 (20)
    Name             : rkisp1_stats
    Function         : V4L2 I/O
    Pad 0x01000015   : 0: Sink
      Link 0x0200002a: from remote pad 0x1000005 of entity 'rkisp1_isp': Data, Enabled, Immutable
Priority: 2
Format Metadata Capture:
    Sample Format   : 'RK1S' (Rockchip ISP1 3A Statistics)
    Buffer Size     : 108
ubuntu@nanopi-m4:~$ /usr/local/bin/v4l2-ctl -d /dev/video9 --all
Driver Info:
    Driver name      : rkisp1
    Card type        : rkisp1_params
    Bus info         : platform:rkisp1
    Driver version   : 5.11.0
    Capabilities     : 0x8c200000
        Metadata Output
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x0c200000
        Metadata Output
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : rkisp1
    Model            : rkisp1
    Serial           : 
    Bus info         : platform:rkisp1
    Media version    : 5.11.0
    Hardware revision: 0x00000000 (0)
    Driver version   : 5.11.0
Interface Info:
    ID               : 0x0300001a
    Type             : V4L Video
Entity Info:
    ID               : 0x00000018 (24)
    Name             : rkisp1_params
    Function         : V4L2 I/O
    Pad 0x01000019   : 0: Source
      Link 0x02000028: to remote pad 0x1000003 of entity 'rkisp1_isp': Data, Enabled, Immutable
Priority: 2
Format Metadata Output:
    Sample Format   : 'RK1P' (Rockchip ISP1 3A Parameters)
    Buffer Size     : 2964
ubuntu@nanopi-m4:~$ /usr/local/bin/v4l2-ctl -d /dev/video10 --all
Driver Info:
    Driver name      : hantro-vpu
    Card type        : rockchip,rk3399-vpu-enc
    Bus info         : platform: hantro-vpu
    Driver version   : 5.11.0
    Capabilities     : 0x84204000
        Video Memory-to-Memory Multiplanar
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04204000
        Video Memory-to-Memory Multiplanar
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : hantro-vpu
    Model            : hantro-vpu
    Serial           : 
    Bus info         : platform: hantro-vpu
    Media version    : 5.11.0
    Hardware revision: 0x00000000 (0)
    Driver version   : 5.11.0
Interface Info:
    ID               : 0x0300000c
    Type             : V4L Video
Entity Info:
    ID               : 0x00000001 (1)
    Name             : rockchip,rk3399-vpu-enc-source
    Function         : V4L2 I/O
    Pad 0x01000002   : 0: Source
      Link 0x02000008: to remote pad 0x1000004 of entity 'rockchip,rk3399-vpu-enc-proc': Data, Enabled, Immutable
Priority: 2
Format Video Capture Multiplanar:
    Width/Height      : 96/32
    Pixel Format      : 'JPEG' (JFIF JPEG)
    Field             : None
    Number of planes  : 1
    Flags             : 
    Colorspace        : JPEG
    Transfer Function : Default
    YCbCr/HSV Encoding: Default
    Quantization      : Default
    Plane 0           :
       Bytes per Line : 0
       Size Image     : 6745
Format Video Output Multiplanar:
    Width/Height      : 96/32
    Pixel Format      : 'YM12' (Planar YUV 4:2:0 (N-C))
    Field             : None
    Number of planes  : 3
    Flags             : 
    Colorspace        : JPEG
    Transfer Function : Default
    YCbCr/HSV Encoding: Default
    Quantization      : Default
    Plane 0           :
       Bytes per Line : 96
       Size Image     : 3072
    Plane 1           :
       Bytes per Line : 48
       Size Image     : 768
    Plane 2           :
       Bytes per Line : 48
       Size Image     : 768

JPEG Compression Controls

            compression_quality 0x009d0903 (int)    : min=5 max=100 step=1 default=50 value=50
ubuntu@nanopi-m4:~$ /usr/local/bin/v4l2-ctl -d /dev/video11 --all
Driver Info:
    Driver name      : hantro-vpu
    Card type        : rockchip,rk3399-vpu-dec
    Bus info         : platform: hantro-vpu
    Driver version   : 5.11.0
    Capabilities     : 0x84204000
        Video Memory-to-Memory Multiplanar
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04204000
        Video Memory-to-Memory Multiplanar
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : hantro-vpu
    Model            : hantro-vpu
    Serial           : 
    Bus info         : platform: hantro-vpu
    Media version    : 5.11.0
    Hardware revision: 0x00000000 (0)
    Driver version   : 5.11.0
Interface Info:
    ID               : 0x0300001a
    Type             : V4L Video
Entity Info:
    ID               : 0x0000000f (15)
    Name             : rockchip,rk3399-vpu-dec-source
    Function         : V4L2 I/O
    Pad 0x01000010   : 0: Source
      Link 0x02000016: to remote pad 0x1000012 of entity 'rockchip,rk3399-vpu-dec-proc': Data, Enabled, Immutable
Priority: 2
Format Video Capture Multiplanar:
    Width/Height      : 48/48
    Pixel Format      : 'NV12' (Y/CbCr 4:2:0)
    Field             : None
    Number of planes  : 1
    Flags             : 
    Colorspace        : JPEG
    Transfer Function : Default
    YCbCr/HSV Encoding: Default
    Quantization      : Default
    Plane 0           :
       Bytes per Line : 48
       Size Image     : 3456
Format Video Output Multiplanar:
    Width/Height      : 48/48
    Pixel Format      : 'MG2S' (MPEG-2 Parsed Slice Data)
    Field             : None
    Number of planes  : 1
    Flags             : 
    Colorspace        : JPEG
    Transfer Function : Default
    YCbCr/HSV Encoding: Default
    Quantization      : Default
    Plane 0           :
       Bytes per Line : 0
       Size Image     : 4608
        mpeg_2_slice_parameters 0x009909fa (unknown): type=103 value=unsupported payload type flags=has-payload
   mpeg_2_quantization_matrices 0x009909fb (unknown): type=104 value=unsupported payload type flags=has-payload
               vp8_frame_header 0x009910d0 (unknown): type=301 value=unsupported payload type flags=has-payload

ubuntu@nanopi-m4:~$ /usr/local/bin/media-ctl -d /dev/media0 -p
Media controller API version 5.11.0

Media device information
------------------------
driver          rkvdec
model           rkvdec
serial          
bus info        platform:rkvdec
hw revision     0x0
driver version  5.11.0

Device topology
- entity 1: rkvdec-source (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video1
    pad0: Source
        -> "rkvdec-proc":0 [ENABLED,IMMUTABLE]

- entity 3: rkvdec-proc (2 pads, 2 links)
            type Node subtype Unknown flags 0
    pad0: Sink
        <- "rkvdec-source":0 [ENABLED,IMMUTABLE]
    pad1: Source
        -> "rkvdec-sink":0 [ENABLED,IMMUTABLE]

- entity 6: rkvdec-sink (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video1
    pad0: Sink
        <- "rkvdec-proc":1 [ENABLED,IMMUTABLE]

ubuntu@nanopi-m4:~$ /usr/local/bin/media-ctl -d /dev/media1 -p
Media controller API version 5.11.0

Media device information
------------------------
driver          rkisp1
model           rkisp1
serial          
bus info        platform:rkisp1
hw revision     0x0
driver version  5.11.0

Device topology
- entity 1: rkisp1_isp (4 pads, 5 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
    pad0: Sink
        [fmt:SRGGB10_1X10/800x600 field:none
         crop.bounds:(0,0)/800x600
         crop:(0,0)/800x600]
        <- "ov13850 1-0010":0 [ENABLED]
    pad1: Sink
        [fmt:unknown/0x0 field:none]
        <- "rkisp1_params":0 [ENABLED,IMMUTABLE]
    pad2: Source
        [fmt:YUYV8_2X8/800x600 field:none
         crop.bounds:(0,0)/800x600
         crop:(0,0)/800x600]
        -> "rkisp1_resizer_mainpath":0 [ENABLED]
        -> "rkisp1_resizer_selfpath":0 [ENABLED]
    pad3: Source
        [fmt:unknown/0x0 field:none]
        -> "rkisp1_stats":0 [ENABLED,IMMUTABLE]

- entity 6: rkisp1_resizer_mainpath (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
    pad0: Sink
        [fmt:YUYV8_2X8/800x600 field:none
         crop.bounds:(0,0)/800x600
         crop:(0,0)/800x600]
        <- "rkisp1_isp":2 [ENABLED]
    pad1: Source
        [fmt:YUYV8_2X8/800x600 field:none]
        -> "rkisp1_mainpath":0 [ENABLED,IMMUTABLE]

- entity 9: rkisp1_resizer_selfpath (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev2
    pad0: Sink
        [fmt:YUYV8_2X8/800x600 field:none
         crop.bounds:(0,0)/800x600
         crop:(0,0)/800x600]
        <- "rkisp1_isp":2 [ENABLED]
    pad1: Source
        [fmt:YUYV8_2X8/800x600 field:none]
        -> "rkisp1_selfpath":0 [ENABLED,IMMUTABLE]

- entity 12: rkisp1_mainpath (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video2
    pad0: Sink
        <- "rkisp1_resizer_mainpath":1 [ENABLED,IMMUTABLE]

- entity 16: rkisp1_selfpath (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video3
    pad0: Sink
        <- "rkisp1_resizer_selfpath":1 [ENABLED,IMMUTABLE]

- entity 20: rkisp1_stats (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video4
    pad0: Sink
        <- "rkisp1_isp":3 [ENABLED,IMMUTABLE]

- entity 24: rkisp1_params (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video5
    pad0: Source
        -> "rkisp1_isp":1 [ENABLED,IMMUTABLE]

- entity 28: ov13850 1-0010 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev3
    pad0: Source
        [fmt:SBGGR10_1X10/2112x1568@10000/300000 field:none]
        -> "rkisp1_isp":0 [ENABLED]

ubuntu@nanopi-m4:~$ /usr/local/bin/media-ctl -d /dev/media2 -p
Media controller API version 5.11.0

Media device information
------------------------
driver          rkisp1
model           rkisp1
serial          
bus info        platform:rkisp1
hw revision     0x0
driver version  5.11.0

Device topology
- entity 1: rkisp1_isp (4 pads, 5 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev4
    pad0: Sink
        [fmt:SBGGR10_1X10/2112x1568 field:none
         crop.bounds:(0,0)/2112x1568
         crop:(0,0)/1920x1080]
        <- "ov13850 2-0010":0 []
    pad1: Sink
        [fmt:unknown/0x0 field:none]
        <- "rkisp1_params":0 [ENABLED,IMMUTABLE]
    pad2: Source
        [fmt:YUYV8_2X8/1920x1080 field:none
         crop.bounds:(0,0)/1920x1080
         crop:(0,0)/1920x1080]
        -> "rkisp1_resizer_mainpath":0 [ENABLED]
        -> "rkisp1_resizer_selfpath":0 []
    pad3: Source
        [fmt:unknown/0x0 field:none]
        -> "rkisp1_stats":0 [ENABLED,IMMUTABLE]

- entity 6: rkisp1_resizer_mainpath (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev5
    pad0: Sink
        [fmt:YUYV8_2X8/1920x1080 field:none
         crop.bounds:(0,0)/1920x1080
         crop:(0,0)/800x600]
        <- "rkisp1_isp":2 [ENABLED]
    pad1: Source
        [fmt:YUYV8_2X8/1920x1080 field:none]
        -> "rkisp1_mainpath":0 [ENABLED,IMMUTABLE]

- entity 9: rkisp1_resizer_selfpath (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev6
    pad0: Sink
        [fmt:YUYV8_2X8/800x600 field:none
         crop.bounds:(0,0)/800x600
         crop:(0,0)/800x600]
        <- "rkisp1_isp":2 []
    pad1: Source
        [fmt:YUYV8_2X8/800x600 field:none]
        -> "rkisp1_selfpath":0 [ENABLED,IMMUTABLE]

- entity 12: rkisp1_mainpath (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video6
    pad0: Sink
        <- "rkisp1_resizer_mainpath":1 [ENABLED,IMMUTABLE]

- entity 16: rkisp1_selfpath (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video7
    pad0: Sink
        <- "rkisp1_resizer_selfpath":1 [ENABLED,IMMUTABLE]

- entity 20: rkisp1_stats (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video8
    pad0: Sink
        <- "rkisp1_isp":3 [ENABLED,IMMUTABLE]

- entity 24: rkisp1_params (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video9
    pad0: Source
        -> "rkisp1_isp":1 [ENABLED,IMMUTABLE]

- entity 28: ov13850 2-0010 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev7
    pad0: Source
        [fmt:SBGGR10_1X10/2112x1568@10000/300000 field:none]
        -> "rkisp1_isp":0 []

ubuntu@nanopi-m4:~$ /usr/local/bin/media-ctl -d /dev/media3 -p
Media controller API version 5.11.0

Media device information
------------------------
driver          hantro-vpu
model           hantro-vpu
serial          
bus info        platform: hantro-vpu
hw revision     0x0
driver version  5.11.0

Device topology
- entity 1: rockchip,rk3399-vpu-enc-source (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video10
    pad0: Source
        -> "rockchip,rk3399-vpu-enc-proc":0 [ENABLED,IMMUTABLE]

- entity 3: rockchip,rk3399-vpu-enc-proc (2 pads, 2 links)
            type Node subtype Unknown flags 0
    pad0: Sink
        <- "rockchip,rk3399-vpu-enc-source":0 [ENABLED,IMMUTABLE]
    pad1: Source
        -> "rockchip,rk3399-vpu-enc-sink":0 [ENABLED,IMMUTABLE]

- entity 6: rockchip,rk3399-vpu-enc-sink (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video10
    pad0: Sink
        <- "rockchip,rk3399-vpu-enc-proc":1 [ENABLED,IMMUTABLE]

- entity 15: rockchip,rk3399-vpu-dec-source (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video11
    pad0: Source
        -> "rockchip,rk3399-vpu-dec-proc":0 [ENABLED,IMMUTABLE]

- entity 17: rockchip,rk3399-vpu-dec-proc (2 pads, 2 links)
             type Node subtype Unknown flags 0
    pad0: Sink
        <- "rockchip,rk3399-vpu-dec-source":0 [ENABLED,IMMUTABLE]
    pad1: Source
        -> "rockchip,rk3399-vpu-dec-sink":0 [ENABLED,IMMUTABLE]

- entity 20: rockchip,rk3399-vpu-dec-sink (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video11
    pad0: Sink
        <- "rockchip,rk3399-vpu-dec-proc":1 [ENABLED,IMMUTABLE]

ubuntu@nanopi-m4:~$

All my attempts to configure the cameras failed, although both cameras have the same bus...

[  917.236061] rkisp1 ff910000.isp0: start pipeline failed -32
[  922.718377] rkisp1 ff910000.isp0: start pipeline failed -32
[  936.024418] rkisp1 ff920000.isp1: start pipeline failed -67
[  940.989432] rkisp1 ff920000.isp1: start pipeline failed -67
[ 1018.300151] rkisp1 ff910000.isp0: start pipeline failed -32
[ 1262.466482] rkisp1 ff920000.isp1: start pipeline failed -67
[ 1729.560950] rkisp1 ff910000.isp0: start pipeline failed -32

Hey @avafinger ,

I don't have another OV13850 but I have a MCAM400, that I wanted to test anyway. The weekend is close and I will try to figure out how to run the dual camera set up, I comment as soon as I found out how.

My current guess is that you probably have not activated the corresponding MIPI port because the second camera doesn't run on the mipi_dphy_rx0 but instead on the mipi_dphy_tx1rx1. If you look here for example here Further down you can see in the pin table that the CSI-1 is connected to the RX0 and the CSI-2 to the TX1RX1. But I believe you said that you have a different board so it might be different for you. If you have the same situation then you can take a look here. This will not work directly as it is quite deprecated, but it might lead you in the right direction.

Good luck and greetings, Sebastian

Update info:

I tested with cam and camera 1 (1-0010) is somehow working, but (2-0010) failed for the reason:

[10098.389695] use of bytesused == 0 is deprecated and will be removed in the future,
[10098.389707] use the actual size instead.
[10109.631551] rockchip-pinctrl pinctrl: pin gpio2-11 already requested by 1-0010; cannot claim for 2-0010
[10109.634124] rockchip-pinctrl pinctrl: pin-75 (2-0010) status -22
[10109.636358] rockchip-pinctrl pinctrl: could not request pin 75 (gpio2-11) from group cif-clkout-a  on device rockchip-pinctrl
[10109.639088] ov13850 2-0010: Error applying setting, reverse things back
[10109.641460] ov13850 2-0010: could not set pins

gpio2-11 is pinctrl-0 = <&cam1_default_pins &cif_clkout_a>;

cif_clkout_a is shared by the two cameras, interesting is the same DT as in FE image and that is working with dual-camera.

I decided to try it with Kernel 4.19.111 and have the same error. Built FE Kernel 4.4.179 and the same thing. Can you spot what is missing?

cam -l

[3:05:12.758777020] [1290]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0+2127-5988661c-dirty (2021-01-08T14:01:36+00:00)
[3:05:12.799473646] [1291] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 2 on pad 0: Inappropriate ioctl for device
[3:05:12.799708435] [1291] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
[3:05:12.809756843] [1291] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 2-0010': Unable to get rectangle 2 on pad 0: Inappropriate ioctl for device
[3:05:12.809818675] [1291] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 2-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
Available cameras:
1: Internal front camera (platform/ff110000.i2c/i2c-1/1-0010 ov13850)
2: Internal front camera (platform/ff120000.i2c/i2c-2/2-0010 ov13850)

cam --camera=1 --capture=20 --stream height=1080,width=1920,pixelformat=YUYV,role=video


[3:05:44.807475825] [1293]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0+2127-5988661c-dirty (2021-01-08T14:01:36+00:00)
[3:05:44.839641698] [1294] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 2 on pad 0: Inappropriate ioctl for device
[3:05:44.839796571] [1294] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
[3:05:44.858181112] [1294] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 2-0010': Unable to get rectangle 2 on pad 0: Inappropriate ioctl for device
[3:05:44.858245861] [1294] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 2-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
Using camera platform/ff110000.i2c/i2c-1/1-0010 ov13850
[3:05:44.862815069] [1293]  INFO Camera camera.cpp:890 configuring streams: (0) 1920x1080-YUYV
[3:05:44.873615096] [1294] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
[3:05:44.873709595] [1294] ERROR CameraSensor camera_sensor.cpp:569 'ov13850 1-0010': Failed to construct camera sensor info: the camera sensor does not report the active area
[3:05:44.873732345] [1294]  WARN RkISP1 rkisp1.cpp:935 Camera sensor information not available
[3:05:44.874007384] [1294]  INFO IPARkISP1 rkisp1.cpp:116 Exposure: 4-1660 Gain: 16-248
Capture 20 frames
11145.067959 (0.00 fps) stream0 seq: 000000 bytesused: 4147200
11145.101262 (30.03 fps) stream0 seq: 000001 bytesused: 4147200
11145.134519 (30.07 fps) stream0 seq: 000002 bytesused: 4147200
11145.167811 (30.04 fps) stream0 seq: 000003 bytesused: 4147200
11145.201099 (30.04 fps) stream0 seq: 000004 bytesused: 4147200
11145.234368 (30.06 fps) stream0 seq: 000005 bytesused: 4147200
11145.267659 (30.04 fps) stream0 seq: 000006 bytesused: 4147200
11145.300939 (30.05 fps) stream0 seq: 000007 bytesused: 4147200
11145.334198 (30.07 fps) stream0 seq: 000008 bytesused: 4147200
11145.434058 (10.01 fps) stream0 seq: 000011 bytesused: 4147200
11145.467337 (30.05 fps) stream0 seq: 000012 bytesused: 4147200
11145.500617 (30.05 fps) stream0 seq: 000013 bytesused: 4147200
11145.533886 (30.06 fps) stream0 seq: 000014 bytesused: 4147200
11145.567176 (30.04 fps) stream0 seq: 000015 bytesused: 4147200
11145.600457 (30.05 fps) stream0 seq: 000016 bytesused: 4147200
11145.633736 (30.05 fps) stream0 seq: 000017 bytesused: 4147200
11145.666994 (30.07 fps) stream0 seq: 000018 bytesused: 4147200
11145.700274 (30.05 fps) stream0 seq: 000019 bytesused: 4147200
11145.733553 (30.05 fps) stream0 seq: 000020 bytesused: 4147200
11145.766833 (30.05 fps) stream0 seq: 000021 bytesused: 4147200

cam --camera=2 --capture=20 --stream height=1080,width=1920,pixelformat=YUYV,role=video

[3:28:16.474749880] [1314]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0+2127-5988661c-dirty (2021-01-08T14:01:36+00:00)
[3:28:16.518692641] [1315] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 2 on pad 0: Inappropriate ioctl for device
[3:28:16.518840222] [1315] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
[3:28:16.529382698] [1315] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 2-0010': Unable to get rectangle 2 on pad 0: Inappropriate ioctl for device
[3:28:16.529452114] [1315] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 2-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
Using camera platform/ff120000.i2c/i2c-2/2-0010 ov13850
[3:28:16.534394064] [1314]  INFO Camera camera.cpp:890 configuring streams: (0) 1920x1080-YUYV
[3:28:16.545428868] [1315] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 2-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
[3:28:16.545518117] [1315] ERROR CameraSensor camera_sensor.cpp:569 'ov13850 2-0010': Failed to construct camera sensor info: the camera sensor does not report the active area
[3:28:16.545540575] [1315]  WARN RkISP1 rkisp1.cpp:935 Camera sensor information not available
[3:28:16.545804239] [1315]  INFO IPARkISP1 rkisp1.cpp:116 Exposure: 4-1660 Gain: 16-248
Capture 20 frames
^CExiting

Attached is a working DT from FE Image with a working dual-camera, just in case you would like to have a look. There, the cif_clkout_a is also shared but the FE kernel does not complain.

camera2.dts.tar.gz

Hey @avafinger ,

I have looked into this issue this weekend and have had the same problem, here is my current attempt: https://github.com/initBasti/Linux_kernel_media_tree_fork/commit/9a8e594a660d67f3790da32f3268bebd3d233526

The issue really seems to be the sharing of the cif_clkout_a GPIO, something must have changed within other parts of the kernel. That is why I started to communicate with the Video 4 Linux team, I will give you feedback as soon as I know more.

Greetings, Sebastian

EDIT: Ah and also on the link that I shared is my current fork of the media_tree, in there I have my current version of the OV13850 driver, where I fixed this error:

[3:28:16.545518117] [1315] ERROR CameraSensor camera_sensor.cpp:569 'ov13850 2-0010': Failed to construct camera sensor info: the camera sensor does not report the active area
[3:28:16.545540575] [1315]  WARN RkISP1 rkisp1.cpp:935 Camera sensor information not available

If you want you can check it out: https://github.com/initBasti/Linux_kernel_media_tree_fork/commit/bebcd38ee58fa9399dbe5abc60e4b0a0fb593518

EDIT-2: Helen Koike from the Kernel Team was just able to tell me why our attempts didn't work. The mainline dphy driver for the rockchip-dphy doesn't support the tx1rx1 dphy and therefore the 2nd camera port. We would have to port that feature from the BSP kernel.

Helen Koike from the Kernel Team was just able to tell me why our attempts didn't work. The mainline dphy driver for the rockchip-dphy doesn't support the tx1rx1 dphy and therefore the 2nd camera port. We would have to port that feature from the BSP kernel.

That's what I suspected, I am not able to run a single camera on CSI-2. Porting this is beyond my skills/time. Is Helen kindly enough to start with that or you plan to start the porting?

We still have the same problem of sharing the pins for the two cameras (multi-camera or dual-camera). It is not possible to share and set the same pins, the kernel will complain. Can you ask Helen how would that be possible? Or how she would approach the problem?

@avafinger , I will look into this issue tomorrow morning and see if it is connected to the pin sharing in some way. I found out on the datasheet of the rk3399 that cif_clkout_a is a pin on the SOC and there is also a cif_clkout_b. (datasheet page 45-46). I am just a bit busy this evening, with another open issue.

Greetings, Sebastian

@initBasti

I had some free time and tested cif_clkout_b and it worked on the legacy kernel. dual-camera running! (I think multi-camera will work also) Thank you.

        cif_clkout_b: cif-clkout-b {
            rockchip,pins = <2 15 3 &pcfg_pull_none>;
        };

and on isp1:

pinctrl-0 = <&cam1_default_pins &cif_clkout_b>;

I am puzzled how FE solved this, they implemented this in the driver.

Anyway, do you have the ov4689 with or without IR Cut filter?

@avafinger That sounds great, can you share the URL of the kernel tree and the .dts file? That would be useful for me when I try to enable this feature in the mainline kernel.

Anyway, do you have the ov4689 with or without IR Cut filter?

With the IR cut filter.

Greetings, Sebastian

@initBasti

Here is my dev kernel for the NanoPi M4 you can have as a reference for the NanoPC-T4.

For reference, Kernel 4.19.111 with dual-camera is here: https://github.com/avafinger/kernel-rockchip-dual-camera

DTS for OV13850 and OV4689: https://github.com/avafinger/kernel-rockchip-dual-camera/blob/dual-camera/arch/arm64/boot/dts/rockchip/rk3399-nanopi4-rkisp1.dtsi

Camera 2 Pin config as you proposed: https://github.com/avafinger/kernel-rockchip-dual-camera/blob/dual-camera/arch/arm64/boot/dts/rockchip/rk3399-nanopi4-common.dtsi#L1217

Hope you are making progress with the porting with mipi_dphy_tx1rx1. Let me know how it goes...

BR, Finger

Hi @initBasti

Did you have any progress with mipi_dphy_tx1rx1 ?

BR, @avafinger

Hey @avafinger ,

Not yet, I am currently finishing the first version of the mainline OV13850 sensor driver. What I currently know is that one of the Kernel developers specializing in Rockchip hardware (mmind), has made an attempt on that project in the past (here), but he was not able to get it to work at that time. Once I have finished the mentioned driver, I will have a look into it and see if I can make it work.

Greetings, Sebastian

Hey @avafinger ,

I have good news, Heiko Stübener has found some time to work on the missing MIPI CSI for the second camera. You can find his work here. I have played around with these changes on my tree at the weekend, but I was not able to make it work for now as there are still some problems with the second instance of the ISP and with my OV4689 driver. But I believe you can try out his tree, as his tree is a lot closer to the Rockchip tree. By the way, if you try out the latest version of my OV13850 driver you will notice that I removed those device tree attributes like module name, module facing, etc. The reason for this change is that they are too specific for a single vendor and therefore do not fit into the driver I intend to bring upstream.

Greetings, Sebastian

Hi @initBasti ,

Thank you for the update.

This weekend i spent a lot of time with Kernel 4.19.161 (Rockchip) with the suggested cif_clkout_b for the second camera despite FE using cif_clkout_a for both cameras. I have not found where in the code the multiplexed pin is allowed. If I am not mistaken, FE 4.19 worked with cif_clkout_a for both cameras, after so many changes and tries I am not sure anymore.

Anyway, FYI, it is all good for 4.19.y , except for the noise. Note that both drivers (ov4689 and ov13850) have been updated. There is better quality but with noise, I will try to get in touch with Rockchip and see if I can get it fixed. The attributes like module name, module facing,etc.. are used by the rkisp engine (bsp), and that works well with cif_clkout_b. I am able to grab images from both cameras at the same time.

For the mainline, i can't say there is noise or not since the image is a little dark, if you work out the 3A to improve it, let me know.

I will try your/his changes soon and report back.

PS: I asked about your OV4689 sensor just because I have both, with and without IR. Interestingly the camera w/o IR gives me a pinkish image (or red tone if you prefer), many sources would say the image is pinkish due to IR being active during daylight, but this camera is without IR. Some sources say it is pinkish precisely because there is no IR on the sensor. It is funny because I can't make IR work, maybe there is a need to activate it with a gpio pin. No doc anywhere.

I think you missed:

&mipi_dsi1 {
    status = "okay";
};

My first try (using Heiko patch), Kernel 5.11.0-rc5:

dual camera Camera 1 OK, camera 2 not OK (seems not powered with cif_clkout_b) This breaks HDMI sound. HDMI still works.

  [   23.747947] hdmi-audio-codec hdmi-audio-codec.5.auto: Only one simultaneous stream supported!
  [   23.748705] hdmi-audio-codec hdmi-audio-codec.5.auto: ASoC: error at snd_soc_dai_startup on i2s-hifi: -22
  [   23.749832] hdmi-audio-codec hdmi-audio-codec.5.auto: Only one simultaneous stream supported!
  [   23.750582] hdmi-audio-codec hdmi-audio-codec.5.auto: ASoC: error at snd_soc_dai_startup on i2s-hifi: -22
  [   23.752157] hdmi-audio-codec hdmi-audio-codec.5.auto: Only one simultaneous stream supported!
  [   23.752907] hdmi-audio-codec hdmi-audio-codec.5.auto: ASoC: error at snd_soc_dai_startup on i2s-hifi: -22
  [   23.754005] hdmi-audio-codec hdmi-audio-codec.5.auto: Only one simultaneous stream supported!
  [   23.754752] hdmi-audio-codec hdmi-audio-codec.5.auto: ASoC: error at snd_soc_dai_startup on i2s-hifi: -22

single camera only in slot CSI-2 ( using cif_clkout_a ) IT Breaks HDMI completely.

  [    9.781262] ov4689 2-0036: driver version: 00.01.01
  [    9.781287] ov4689 2-0036: could not get module information!
  [    9.785462] ov4689 2-0036: Unexpected sensor id(000000), ret(-5)

What is your experience?

Module                  Size  Used by
algif_hash             20480  1
algif_skcipher         20480  1
af_alg                 32768  6 algif_hash,algif_skcipher
bnep                   28672  2
cpufreq_powersave      16384  0
cpufreq_conservative    16384  0
crct10dif_ce           20480  1
hantro_vpu             90112  0
btsdio                 20480  0
rockchip_rga           28672  0
snd_soc_spdif_tx       16384  1
snd_soc_simple_card    20480  0
rockchip_vdec          28672  0
rockchip_isp1          73728  0
brcmfmac              249856  0
panfrost               69632  0
snd_soc_simple_card_utils    24576  1 snd_soc_simple_card
v4l2_h264              16384  2 rockchip_vdec,hantro_vpu
rockchipdrm           131072  0
videobuf2_vmalloc      20480  2 rockchip_isp1,hantro_vpu
videobuf2_dma_sg       24576  1 rockchip_rga
gpu_sched              32768  1 panfrost
videobuf2_dma_contig    24576  3 rockchip_vdec,rockchip_isp1,hantro_vpu
v4l2_mem2mem           40960  3 rockchip_vdec,hantro_vpu,rockchip_rga
ov4689                 24576  0
videobuf2_memops       20480  3 videobuf2_vmalloc,videobuf2_dma_contig,videobuf2_dma_sg
brcmutil               20480  1 brcmfmac
realtek                28672  1
analogix_dp            40960  1 rockchipdrm
videobuf2_v4l2         32768  5 rockchip_vdec,rockchip_isp1,hantro_vpu,rockchip_rga,v4l2_mem2mem
cfg80211              401408  1 brcmfmac
v4l2_fwnode            24576  2 rockchip_isp1,ov4689
videobuf2_common       57344  6 rockchip_vdec,rockchip_isp1,videobuf2_v4l2,hantro_vpu,rockchip_rga,v4l2_mem2mem
dwmac_rk               28672  0
dw_mipi_dsi            20480  1 rockchipdrm
snd_soc_rockchip_i2s    20480  2
videodev              258048  9 rockchip_vdec,v4l2_fwnode,rockchip_isp1,videobuf2_v4l2,hantro_vpu,ov4689,rockchip_rga,videobuf2_common,v4l2_mem2mem
stmmac_platform        20480  1 dwmac_rk
dw_hdmi                45056  1 rockchipdrm
snd_soc_rockchip_pcm    16384  1 snd_soc_rockchip_i2s
snd_soc_rockchip_spdif    16384  2
stmmac                188416  2 stmmac_platform,dwmac_rk
cec                    57344  1 dw_hdmi
hci_uart               57344  0
rtc_rk808              16384  1
mc                     53248  8 rockchip_vdec,videodev,rockchip_isp1,videobuf2_v4l2,hantro_vpu,ov4689,videobuf2_common,v4l2_mem2mem
drm_kms_helper        249856  4 dw_mipi_dsi,rockchipdrm,dw_hdmi,analogix_dp
btbcm                  24576  1 hci_uart
snd_soc_rt5651         94208  1
rockchip_thermal       24576  0
rockchip_saradc        16384  0
snd_soc_rl6231         16384  1 snd_soc_rt5651
pcs_xpcs               20480  1 stmmac
industrialio_triggered_buffer    16384  1 rockchip_saradc
kfifo_buf              16384  1 industrialio_triggered_buffer
pcie_rockchip_host     20480  0
drm                   569344  8 gpu_sched,drm_kms_helper,dw_mipi_dsi,rockchipdrm,dw_hdmi,panfrost,analogix_dp
ip_tables              32768  0
x_tables               40960  1 ip_tables
ipv6                  442368  48

@initBasti

I have noticed Heiko has patches for the HDMI to DSI: https://github.com/mmind/linux-rockchip/commit/5044a21e2a1af01511ff15cf3f250468d9d59428

Do you think it is relevant?

Second attempt, kernel 5.11-rc6 (cif_clkout_b):

[    9.812773] ov4689 1-0036: driver version: 00.01.01
[    9.812786] ov4689 1-0036: could not get module information!
[    9.815351] ov4689 1-0036: Detected OV004688 sensor
[    9.815982] ov4689 2-0036: driver version: 00.01.01
[    9.815999] ov4689 2-0036: could not get module information!
[    9.816042] ov4689 2-0036: Failed to get reset-gpios
[    9.816055] ov4689 2-0036: Failed to get pwdn-gpios
[    9.818417] ov4689 2-0036: Unexpected sensor id(000000), ret(-5)

Note this :

[    9.816042] ov4689 2-0036: Failed to get reset-gpios
[    9.816055] ov4689 2-0036: Failed to get pwdn-gpios

And Heiko Hack for the ov13850: https://github.com/mmind/linux-rockchip/commit/9641b7a09a748b9ba7960d1301b60c228058d4a7

Unfortunately, HDMI is gone....

Can you check with Heiko if he is using his board headless?

Complete boot log: https://gist.github.com/avafinger/7b08a83ca596d6ad5af2033720d04664#file-kernel-5-11-0-rc6-dual-camera

Without the Heiko's patch (single camera): https://gist.github.com/avafinger/7b08a83ca596d6ad5af2033720d04664

ubuntu@nanopi-m4:~$ media-ctl -d /dev/media1 -p|grep enti
- entity 1: rkisp1_isp (4 pads, 5 links)
- entity 6: rkisp1_resizer_mainpath (2 pads, 2 links)
- entity 9: rkisp1_resizer_selfpath (2 pads, 2 links)
- entity 12: rkisp1_mainpath (1 pad, 1 link)
- entity 16: rkisp1_selfpath (1 pad, 1 link)
- entity 20: rkisp1_stats (1 pad, 1 link)
- entity 24: rkisp1_params (1 pad, 1 link)
- entity 28: ov4689 1-0036 (1 pad, 1 link)
ubuntu@nanopi-m4:~$ media-ctl -d /dev/media2 -p|grep enti
- entity 1: rkisp1_isp (4 pads, 0 link)
- entity 6: rkisp1_resizer_mainpath (2 pads, 0 link)
- entity 9: rkisp1_resizer_selfpath (2 pads, 0 link)
- entity 12: rkisp1_mainpath (1 pad, 0 link)
- entity 16: rkisp1_selfpath (1 pad, 0 link)
- entity 20: rkisp1_stats (1 pad, 0 link)
- entity 24: rkisp1_params (1 pad, 0 link)

Fixed the gpio mistake, dual-camera shot, unfortunately, it breaks hdmi (drm) on the mainline kernel. Maybe you have better luck.

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
[    0.000000] Linux version 5.11.0-rc6+ (alex@svn) (aarch64-linux-gnu-gcc (Linaro GCC 7.5-2019.12-rc1) 7.5.0, GNU ld (Linaro_Binutils-2019.12-rc1) 2.28.2.20170706) #2 SMP PREEMPT Tue Feb 2 19:43:41 -03 2021
[   10.162529] mc: Linux media interface: v0.10
[   10.958473] videodev: Linux video capture interface: v2.00
[   11.378146] ov13850 1-0010: driver version: 00.01.01
[   11.378167] ov13850 1-0010: could not get module information!
[   11.380785] ov13850 1-0010: Unexpected sensor id(000000), ret(-5)
[   11.382015] ov13850 2-0010: driver version: 00.01.01
[   11.382034] ov13850 2-0010: could not get module information!
[   11.384354] ov13850 2-0010: Unexpected sensor id(000000), ret(-5)
[   11.482671] ov4689 1-0036: driver version: 00.01.01
[   11.482684] ov4689 1-0036: could not get module information!
[   11.485346] ov4689 1-0036: Detected OV004688 sensor
[   11.485638] ov4689 2-0036: driver version: 00.01.01
[   11.485652] ov4689 2-0036: could not get module information!
[   11.487807] ov4689 2-0036: Unexpected sensor id(000000), ret(-5)

Hey @avafinger ,

first of all sorry for the delayed response, I have been working on this issue and I am finally at a point where it works for me. I will first talk about some of the points you made in your last messages.

Anyway, FYI, it is all good for 4.19.y , except for the noise. Note that both drivers (ov4689 and ov13850) have been updated. There is better quality but with noise, I will try to get in touch with Rockchip and see if I can get it fixed.

I have noticed a lot of noise as well, but I believe those are also fixed with the 3A algorithms. The reason for this belief is that I have tried both cameras on the official friendly elec image, where they have pre-installed a video capture software, which produces a quite clean output.

The attributes like module name, module facing, etc.. are used by the rkisp engine (BSP)

Ok, well then that means that my driver is not compatible with that kernel. As the rkisp1 engine from the mainline kernel doesn't expect such arguments.

For the mainline, I can't say there is noise or not since the image is a little dark, if you work out the 3A to improve it, let me know.

This is on my todo list for a few months now ;) I will message you as soon as I am at a point where you can test. Could you provide me an Email address?

Do you know what is the purpose of this hack? mmind/linux-rockchip@9641b7a

Nope, and I didn't require it as well.

I think you missed:

&mipi_dsi1 { status = "okay"; };

True this was one of the issues that I encountered, thank you for your good eyes :).

I have noticed Heiko has patches for the HDMI to DSI: mmind/linux-rockchip@5044a21 Do you think it is relevant?

I have not played around with HDMI until now and I just noticed, that there might be problems. What I have experienced with the setup that I explain further below is that everything works if I have no HDMI devices connected during boot. The media device graph can be created and the cam command finishes everything until the capture but then it halts. This means that the TX/RX seems to be reserved for the TX and therefore not available for the RX. But the rkisp1 has 3 different PHY a TX, an RX, and a TX/RX, so maybe there is a way to take the pure TX for the HDMI out. I am not completely sure about this theory though, I will mention this is Heiko's upstream patch series mentioned below.

I have been able to capture from both cameras at the same time using the following setup: My patches for the OV13850, OV4689, and the device tree sources: https://github.com/initBasti/Linux_kernel_media_tree_fork/commits/dual_cam_setup

Heiko Stübner's upstream patch series: https://patchwork.kernel.org/project/linux-media/list/?series=426281

The image of the OV4689 looks very bad and has big green patches I currently investigate if it is a problem with the PHY, ISP instance, or the camera driver. If possible please write your test results directly as a mail response to the patch series of Heiko, so that he can work with it.

Greetings, Sebastian

The image of the OV4689 looks very bad

Do you mean for the second camera?

The image of the OV4689 looks very bad

Do you mean for the second camera?

I have tested if it is the camera or the Image Signal Processor instance, by switching their ports so that the OV13850 is connected to the 2nd ISP instance and the OV4689 to the 1st instance. And the image was still bad so it's not the second camera but the failure is specifically at the OV4689 (I am really not surprised as I haven't worked a lot on that driver so far).

I get a good image but with the wrong window size or cropped, certainly something wrong. The sample here is from mjpg streamer that should be streaming 1920x1080 and I get a portion of the image resized (maximized, upper left). The real image is in the frames-120.mp4 (link below). mjpg_1920x1080

Maybe you can spot what is wrong:

media-ctl --device "platform:rkisp1" --reset
# connect pad 0 (sink) of the ISP with pad 0 of the camera and enable the link
media-ctl --device "platform:rkisp1" --links "'ov4689 1-0036':0 -> 'rkisp1_isp':0 [1]"
# create a link between the selfpath (preview) and the ISP output on pad 2 (source), but keep it deactivated
media-ctl --device "platform:rkisp1" --links "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [0]"
# create a link between the mainpath and the ISP output on pad 2 (source) and enable the link
media-ctl --device "platform:rkisp1" --links "'rkisp1_isp':2 -> 'rkisp1_resizer_mainpath':0 [1]"

# Set the video format on the camera (this is very dependent on your camera)
media-ctl --device "platform:rkisp1" --set-v4l2 '"ov4689 1-0036":0 [fmt:SBGGR10_1X10/2688x1520]'
# Set the input video format for the ISP, this must match the video format of the camera, crop it down to 1920x1080
media-ctl --device "platform:rkisp1" --set-v4l2 '"rkisp1_isp":0 [fmt:SBGGR10_1X10/2688x1520 crop: (0,0)/1920x1080]'
# Set the output video format of the ISP, the maximum size was propagated from the sink pad, and the format size is taken from the crop format
media-ctl --device "platform:rkisp1" --set-v4l2 '"rkisp1_isp":2 [fmt:YUYV8_2X8/1920x1080 crop: (0,0)/1920x1080]'

# Set the input format for the mainpath resizer
media-ctl --device "platform:rkisp1" --set-v4l2 '"rkisp1_resizer_mainpath":0 [fmt:YUYV8_2X8/1920x1080]'
# Set the output format for the mainpath resizer
media-ctl --device "platform:rkisp1" --set-v4l2 '"rkisp1_resizer_mainpath":1 [fmt:YUYV8_2X8/1920x1080]'

# Configure the format at the mainpath DMA-engine, which is the point that is accessed by user-space
v4l2-ctl --media-bus-info "platform:rkisp1" --device "rkisp1_mainpath" --set-fmt-video "width=1920,height=1080,pixelformat=YUYV"

Here is a video (120 frames) that should be 1920x1080 but got a 1920x1520 frame size with the real image smaller, resized, or cropped. Where you see Blue is Red. The camera is w/o IR cut.

https://drive.google.com/file/d/14UkhxQx7mA1w9xo748WxmakIGM-70qHd/view?usp=sharing

No progress with dual-camera, my kernel is based on mainline kernel.

Hey @avafinger ,

No progress with dual-camera, my kernel is based on mainline kernel.

That is interesting when you use the setup that I described here:

I have been able to capture from both cameras at the same time using the following setup: My patches for the OV13850, OV4689, and the device tree sources: https://github.com/initBasti/Linux_kernel_media_tree_fork/commits/dual_cam_setup

Heiko Stübner's upstream patch series: https://patchwork.kernel.org/project/linux-media/list/?series=426281

Then you cannot stream with two cameras? Hmmm, I updated this repository maybe you can try to build a new kernel with the patches that I use.

The setup that you use looks correct but I am not very familiar with mjpeg streamer. It looks like the top left crop that you created during media device configuration is cropped down even further at some point. I believe that the color issues are probably related to the 3A algorithms again.

Greetings, Sebastian

Hi @initBasti ,

I am not very familiar with mjpeg streamer

mjpeg streamer just grabs the yuv frame and sends it to the web client encoded in jpeg format. No changes during this process. The yuv frame comes magnified from the camera. I was expecting I did something silly on the configuration.

Anyway, my setup is a bit different than yours, I will try a few things I have in mind. I have everything working with this kernel, hdmi,panfrost, wlan,eth,bt,rt5651, and hdmi-sound and would like not to lose these features.

related to the 3A algorithms again

This only happens with the sensor w/o IR cut. Works fine the one with IR.

I think I have a clue, I will test a few things.

Thanks, BR

Hi @initBasti

In my last attempt, i fixed my issues and now I have the image with the correct size but I found an interesting thing, i tested my changes with those two cameras (below). The left camera does not work, the right camera works (with HDMI working).

The left camera gives me a timeout during the select, although i can get info from the camera and set parameters. So, they behave differently when there is IR cut filter, the funny thing is that you seem to have the left one, right?

Here is the final output image of the right camera:: ov4689_192x1080

@avafinger Good to hear that you had some progress, looking into the OV4689 is on my todo list but not very far up, so it might take a while before I can look into it (~3months probably), when I look into it I will buy myself the other version as well. By the way, Heiko has posted a new version of his patch set where he fixes the HDMI on boot issue (https://patchwork.kernel.org/project/linux-media/list/?series=431387) Greetings, Sebastian

@initBasti Moved to mainline kernel 5.11.0 and now I am able to make dual ov13850 work. dual ov4689 work if w/o ir cut. Good images for both. The only drawback is I can mjpg stream video from CSI-2 (ov1380 / ov5689) but not able to configure the capture options on CSI-1 if in dual mode. Cam app works on both dual cameras. I will release one raw image and that's it for now. If you have the opportunity to work with ov4689 w/ir cut (I get timeout) i will come back and try again. Thank you!

@initBasti FYI, i just got a new ov4689 w/IR cut and i can now grab the images. In conclusion, I somehow damaged the sensor while switching sensors. Anyway, the latest FE kernel has some code for dealing with IR cut, have you tried to enable IR for the night shots?

Meanwhile, I got some ov5647 sensors with IR (auto cut) for 4B+, and as soon as I get the 4B+ board will try some experiments. Do you know how is the state of ov5647 in mainline? I was expecting to get some rk3566 for this but could not find any specification for the connector. I have seen you have 4B, have you tried any sensor on that?

ov5647

Hi @initBasti

It's been a while since your last post, I wonder if you worked on libcamera (or ov4689) and can get a decent image from ov4689

Hey @avafinger ,

Oh, I am terribly sorry for replying so late. I am currently working on an automated camera test workflow for libcamera (currently for the combinations Raspberry Pi + IMX219, Raspberry Pi + IMX477, RockPi + IMX219). And I will probably work on this for the following months as well, as it is quite a big project. Regarding your questions, the OV4689 is currently quite low on my list as different tasks on libcamera have been pilling up. Also, the OV13850 project is currently a bit difficult as the leaked documentation online doesn't match the current version of the device. I have tried to contact Omnivision, but well ... they are not interested in my work. This means that I have a hard time to fulfill the requests that I have received in the review of my kernel upstream patch. And the situation is even worse for the OV4689 :/.

I have not yet worked with OV5647, but as I build camera test boxes for the libcamera project with different platforms + camera combinations, I will probably work with it in the near future. Again sorry for not being able to respond earlier.

Greetings, Sebastian

@initBasti

No problems. Thank you for your reply.

I just tested ov4689, kernel 5.13, with the latest libcamera but got a few more errors than the previous version. I don't know how to fix it, maybe you could give some hints or directions, or it is just my pipeline setup (cam don't rely on the manual pipeline, right?).

V4L2_CID_CAMERA_ORIENTATION
[41:24:39.211650305] [63859]  WARN CameraSensor camera_sensor.cpp:197 'ov4689 1-0036': Recommended V4L2 control 0x009a0922 not supported

[41:24:39.211769595] [63859] ERROR V4L2 v4l2_subdevice.cpp:286 'ov4689 1-0036': Unable to get rectangle 2 on pad 0: Inappropriate ioctl for device
[41:24:39.211827053] [63859]  WARN CameraSensor camera_sensor.cpp:224 'ov4689 1-0036': The PixelArraySize property has been defaulted to 2688x1520

[41:24:39.211874594] [63859] ERROR V4L2 v4l2_subdevice.cpp:286 'ov4689 1-0036': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device

V4L2_SEL_TGT_CROP_BOUNDS && V4L2_SEL_TGT_CROP_DEFAULT
[41:24:39.211913677] [63859]  WARN CameraSensor camera_sensor.cpp:235 'ov4689 1-0036': The PixelArrayActiveAreas property has been defaulted to (0x0)/2688x1520
[41:24:39.211983093] [63859] ERROR V4L2 v4l2_subdevice.cpp:286 'ov4689 1-0036': Unable to get rectangle 0 on pad 0: Inappropriate ioctl for device
[41:24:39.212067383] [63859]  WARN CameraSensor camera_sensor.cpp:243 'ov4689 1-0036': Failed to retrieve the sensor crop rectangle
[41:24:39.212100925] [63859]  WARN CameraSensor camera_sensor.cpp:249 'ov4689 1-0036': The sensor kernel driver needs to be fixed
[41:24:39.212352797] [63859]  WARN CameraSensor camera_sensor.cpp:251 'ov4689 1-0036': See Documentation/sensor_driver_requirements.rst in the libcamera sources for more information
[41:24:39.226221380] [63859]  WARN CameraSensorProperties camera_sensor_properties.cpp:123 No static properties available for 'ov4689'
[41:24:39.226281171] [63859]  WARN CameraSensorProperties camera_sensor_properties.cpp:125 Please consider updating the camera sensor properties database

V4L2_CID_CAMERA_ORIENTATION
[41:24:39.226308587] [63859]  WARN CameraSensor camera_sensor.cpp:403 'ov4689 1-0036': Failed to retrieve the camera location

About the ov5647, i have read it is on pair with RPI but not able to test it. I have not received my 4B+, that will take some time...

Regards, avafinger

Hey @avafinger ,

so the problem boils down to the missing IOCTL for getting the current selection, I had the same problem with the OV13850 and I was able to solve it by adding the following function. There are 4 possible selections the current CROP window (V4L2_SEL_TGT_CROP), the NATIVE size of the camera image (V4L2_SEL_TGT_NATIVE_SIZE) and the bounds for cropping (V4L2_SEL_TGT_CROP_BOUNDS) as well as V4L2_SEL_TGT_CROP_DEFAULT which is often the same as the bounds.

I have declared those macros here and I got the values from the documentation (this part could be tricky for the OV4689 and don't expect support from Omnivision, maybe you can find out what works by trying). For V4L2_SEL_TGT_CROP, I have implemented a function which returns the current crop window from either the device directly or from the try format, which is only stored on the filehandle of the sub-device (used for cases when multiple applications try to work with a camera simultaneously). Libcamera only cares about the active window so you could skip the try format if you only desire to work with libcamera for now. I needed to add cur_mode to the struct ov13850 structure and add the default crop windows within struct ov13850_mode supported_modes[]. For the ov13850 it is the same for both as the first mode (2112x1568) is binned meaning multiple pixels are combined to a single superpixel to reduce noise, so it actually refers to the same size.

I hope this helps.

Greetings, Sebastian

@initBasti

Thank you, I missed that work but yesterday i had the following on my tests:

static int ov4689_get_selection(struct v4l2_subdev *sd,
                struct v4l2_subdev_pad_config *cfg,
                struct v4l2_subdev_selection *sel)
{
        struct ov4689 *ov4689 = to_ov4689(sd);

        switch (sel->target) {
        case V4L2_SEL_TGT_CROP_DEFAULT:
        case V4L2_SEL_TGT_CROP_BOUNDS:
        case V4L2_SEL_TGT_CROP:
        case V4L2_SEL_TGT_NATIVE_SIZE:
                sel->r.left = 0;
                sel->r.top = 0;
                sel->r.width = ov4689->cur_mode->width;
                sel->r.height = ov4689->cur_mode->height;
                return 0;
        default:
                return -EINVAL;
        }
}

But latest libcamera seems to require V4L2_CID_CAMERA_ORIENTATION.

sorry to hijack this conversation... @initBasti @avafinger did one of you already get a NanoPC-T6? The MCAM400 was made for the T4, but I'm currently trying to get it to work in dual-mode with the T6 which is a real power horse... support is a mess so far unfortunately...

My board is on the way. I expect to get it at the end of this month, hopefully. I need the encoder and decoder working so first i will get it to work with BSP...

I digged out the device tree from the rockpro64 I fixed with the help of @initBasti some time ago for the OV13850... trying some new tweaks tomorrow morning to see if that works for either the OV13850 or the MCAM400...

what we have so far is mpp working and @philfleck is looking into this based on the 5.10.y kernel blob which seems to work already quite ok... funny that there is around 10 people only which you run across virtually all the time :-)

https://github.com/rockchip-linux/mpp/issues/253#issuecomment-1744816799

Yeah. Someone already got that working, see here: https://www.friendlyelec.com/Forum/viewtopic.php?f=81&p=12808&sid=e90a643db39f660598779f55c0098e80#p12174

I think you need to tweak a bit for the dual sensor. Post your findings / changes.

Encoder and Decoder are working great for NV12, but YUYV is a trouble. https://forum.radxa.com/t/imx415-npu-demo-on-rock-5b/11319/24

you are my hero! I did not find that post when I was googling for hours... will check out tomorrow immediately... thx a lot

so quick reply... it does not work yet... either there is an essential bit missing in the description/files, or I'm still doing something wrong... will keep you posted...

Did you get at least the video devices?

ls -la /dev/video*

hi, yes, a ton actually,

crw-rw---- 1 root video 81,  0 Oct  4 10:51 /dev/video0
crw-rw---- 1 root video 81,  1 Oct  4 10:51 /dev/video1
crw-rw---- 1 root video 81, 10 Oct  4 10:51 /dev/video10
crw-rw---- 1 root video 81, 11 Oct  4 10:51 /dev/video11
crw-rw---- 1 root video 81, 12 Oct  4 10:51 /dev/video12
crw-rw---- 1 root video 81, 13 Oct  4 10:51 /dev/video13
crw-rw---- 1 root video 81, 14 Oct  4 10:51 /dev/video14
crw-rw---- 1 root video 81, 15 Oct  4 10:51 /dev/video15
crw-rw---- 1 root video 81, 16 Oct  4 10:51 /dev/video16
crw-rw---- 1 root video 81, 17 Oct  4 10:51 /dev/video17
crw-rw---- 1 root video 81, 18 Oct  4 10:51 /dev/video18
crw-rw---- 1 root video 81, 19 Oct  4 10:51 /dev/video19
crw-rw---- 1 root video 81,  2 Oct  4 10:51 /dev/video2
crw-rw---- 1 root video 81, 20 Oct  4 10:51 /dev/video20
crw-rw---- 1 root video 81, 21 Oct  4 10:51 /dev/video21
crw-rw---- 1 root video 81, 22 Oct  4 10:51 /dev/video22
crw-rw---- 1 root video 81, 23 Oct  4 10:51 /dev/video23
crw-rw---- 1 root video 81, 24 Oct  4 10:51 /dev/video24
crw-rw---- 1 root video 81, 25 Oct  4 10:51 /dev/video25
crw-rw---- 1 root video 81, 26 Oct  4 10:51 /dev/video26
crw-rw---- 1 root video 81, 27 Oct  4 10:51 /dev/video27
crw-rw---- 1 root video 81, 28 Oct  4 10:51 /dev/video28
crw-rw---- 1 root video 81, 29 Oct  4 10:51 /dev/video29
crw-rw---- 1 root video 81,  3 Oct  4 10:51 /dev/video3
crw-rw---- 1 root video 81, 30 Oct  4 10:51 /dev/video30
crw-rw---- 1 root video 81, 31 Oct  4 10:51 /dev/video31
crw-rw---- 1 root video 81, 32 Oct  4 10:51 /dev/video32
crw-rw---- 1 root video 81, 33 Oct  4 10:51 /dev/video33
crw-rw---- 1 root video 81, 34 Oct  4 10:51 /dev/video34
crw-rw---- 1 root video 81, 35 Oct  4 10:51 /dev/video35
crw-rw---- 1 root video 81, 36 Oct  4 10:51 /dev/video36
crw-rw---- 1 root video 81, 37 Oct  4 10:51 /dev/video37
crw-rw---- 1 root video 81, 38 Oct  4 10:51 /dev/video38
crw-rw---- 1 root video 81, 39 Oct  4 10:51 /dev/video39
crw-rw---- 1 root video 81,  4 Oct  4 10:51 /dev/video4
crw-rw---- 1 root video 81, 40 Oct  4 10:51 /dev/video40
crw-rw---- 1 root video 81,  5 Oct  4 10:51 /dev/video5
crw-rw---- 1 root video 81,  6 Oct  4 10:51 /dev/video6
crw-rw---- 1 root video 81,  7 Oct  4 10:51 /dev/video7
crw-rw---- 1 root video 81,  8 Oct  4 10:51 /dev/video8
crw-rw---- 1 root video 81,  9 Oct  4 10:51 /dev/video9

and 4 media devices that link all of those:

https://www.dropbox.com/scl/fi/5e2evasrczfkznhyp55xx/graph0.pdf?rlkey=cv9ycjosjq1glduoemta2f2vv&dl=0 https://www.dropbox.com/scl/fi/31z11c5xrgj8qdj7arh5n/graph1.pdf?rlkey=mwmxcefkgexfxmttb3q90r5k0&dl=0 https://www.dropbox.com/scl/fi/3gbyknh3f27espy0sb1pa/graph2.pdf?rlkey=kwddds004u8qcwkehamb4il1s&dl=0 https://www.dropbox.com/scl/fi/7karx6d673vk339ylrwpn/graph3.pdf?rlkey=704on7pz86qbk9ld13gdu5mdt&dl=0

but the issue is that the kernel log is full of issues and I'm not sure if it happens because I compiled the sensor driver as a module or not...

[    9.833240] rockchip-csi2-dphy3: No link between dphy and sensor
[    9.833262] rkcif-mipi-lvds4: rkcif_update_sensor_info: stream[2] get remote terminal sensor failed!
[    9.833272] stream_cif_mipi_id2: update sensor info failed -19
[    9.833894] rockchip-csi2-dphy3: No link between dphy and sensor
[    9.833907] rockchip-csi2-dphy3: No link between dphy and sensor
[    9.833915] rkcif-mipi-lvds4: rkcif_update_sensor_info: stream[2] get remote terminal sensor failed!
[    9.833920] rkcif-mipi-lvds4: rkcif_update_sensor_info: stream[0] get remote terminal sensor failed!
[    9.833923] rkcif_scale_ch2: update sensor info failed -19
[    9.833924] rkcif_scale_ch0: update sensor info failed -19
[    9.834133] rockchip-csi2-dphy3: No link between dphy and sensor
[    9.834139] rkcif-mipi-lvds4: rkcif_update_sensor_info: stream[0] get remote terminal sensor failed!
[    9.834143] stream_cif_mipi_id0: update sensor info failed -19
[    9.836074] rockchip-csi2-dphy3: No link between dphy and sensor
[    9.836091] rkcif-mipi-lvds4: rkcif_update_sensor_info: stream[3] get remote terminal sensor failed!
[    9.836096] rkcif_scale_ch3: update sensor info failed -19
[    9.836107] rockchip-csi2-dphy3: No link between dphy and sensor
[    9.836115] rkcif-mipi-lvds4: rkcif_update_sensor_info: stream[1] get remote terminal sensor failed!
[    9.836119] stream_cif_mipi_id1: update sensor info failed -19
[    9.836143] rockchip-csi2-dphy0: No link between dphy and sensor
[    9.836182] rkcif-mipi-lvds2: rkcif_update_sensor_info: stream[2] get remote terminal sensor failed!
[    9.836189] stream_cif_mipi_id2: update sensor info failed -19
[    9.836357] rockchip-csi2-dphy0: No link between dphy and sensor
[    9.836366] rkcif-mipi-lvds2: rkcif_update_sensor_info: stream[2] get remote terminal sensor failed!
[    9.836370] rkcif_tools_id2: update sensor info failed -19
[    9.836425] rockchip-csi2-dphy3: No link between dphy and sensor
[    9.836440] rkcif-mipi-lvds4: rkcif_update_sensor_info: stream[0] get remote terminal sensor failed!
[    9.836447] rkcif_tools_id0: update sensor info failed -19
[    9.836622] rockchip-csi2-dphy3: No link between dphy and sensor
[    9.836642] rkcif-mipi-lvds4: rkcif_update_sensor_info: stream[3] get remote terminal sensor failed!
[    9.836645] stream_cif_mipi_id3: update sensor info failed -19
[    9.837209] rockchip-csi2-dphy0: No link between dphy and sensor
[    9.837224] rkcif-mipi-lvds2: rkcif_update_sensor_info: stream[0] get remote terminal sensor failed!
[    9.837228] stream_cif_mipi_id0: update sensor info failed -19
[    9.837801] rockchip-csi2-dphy0: No link between dphy and sensor
[    9.837815] rkcif-mipi-lvds2: rkcif_update_sensor_info: stream[1] get remote terminal sensor failed!
[    9.837820] rkcif_scale_ch1: update sensor info failed -19
[    9.838433] rockchip-csi2-dphy0: No link between dphy and sensor
[    9.838447] rkcif-mipi-lvds2: rkcif_update_sensor_info: stream[2] get remote terminal sensor failed!
[    9.838451] rkcif_scale_ch2: update sensor info failed -19
[    9.838481] rockchip-csi2-dphy0: No link between dphy and sensor
[    9.838489] rkcif-mipi-lvds2: rkcif_update_sensor_info: stream[0] get remote terminal sensor failed!
[    9.838492] rkcif_tools_id0: update sensor info failed -19
[    9.838613] rockchip-csi2-dphy0: No link between dphy and sensor
[    9.838618] rkcif-mipi-lvds2: rkcif_update_sensor_info: stream[0] get remote terminal sensor failed!
[    9.838622] rkcif_scale_ch0: update sensor info failed -19
[    9.839485] rockchip-csi2-dphy3: No link between dphy and sensor
[    9.839528] rkcif-mipi-lvds4: rkcif_update_sensor_info: stream[1] get remote terminal sensor failed!
[    9.839537] rkcif_scale_ch1: update sensor info failed -19
[    9.839741] rockchip-csi2-dphy0: No link between dphy and sensor
[    9.839755] rkcif-mipi-lvds2: rkcif_update_sensor_info: stream[3] get remote terminal sensor failed!
[    9.839758] stream_cif_mipi_id3: update sensor info failed -19
[    9.840172] rockchip-csi2-dphy3: No link between dphy and sensor
[    9.840189] rkcif-mipi-lvds4: rkcif_update_sensor_info: stream[2] get remote terminal sensor failed!
[    9.840193] rkcif_tools_id2: update sensor info failed -19
[    9.840434] rockchip-csi2-dphy0: No link between dphy and sensor
[    9.840462] rkcif-mipi-lvds2: rkcif_update_sensor_info: stream[1] get remote terminal sensor failed!
[    9.840470] stream_cif_mipi_id1: update sensor info failed -19
[    9.841228] rockchip-csi2-dphy3: No link between dphy and sensor
[    9.841245] rkcif-mipi-lvds4: rkcif_update_sensor_info: stream[1] get remote terminal sensor failed!
[    9.841248] rkcif_tools_id1: update sensor info failed -19
[    9.841867] rockchip-csi2-dphy0: No link between dphy and sensor
[    9.841882] rkcif-mipi-lvds2: rkcif_update_sensor_info: stream[3] get remote terminal sensor failed!
[    9.841886] rkcif_scale_ch3: update sensor info failed -19
[    9.848156] rockchip-csi2-dphy0: No link between dphy and sensor
[    9.848176] rkcif-mipi-lvds2: rkcif_update_sensor_info: stream[1] get remote terminal sensor failed!
[    9.848183] rkcif_tools_id1: update sensor info failed -19

Which kernel version?

Rockchip-Kernel from NanoPC-T6 WIKI - https://github.com/friendlyarm/kernel-rockchip nanopi5-v5.10.y_opt branch - which is 5.10.160 or so... unfortunately the post did not reveal which kernel version was used, but I suspect it is this kernel...

PS, did not make a difference to use the OV as a module or not... PPS, OH, it did make a difference... let me check.... PPPS: It worked! So it seems that the OV4689 has to be compiled into the kernel... let me see if I can move on from there...

Did you attach the 2 sensors?

Yes… both seem to work, but i just grabbed yuv images yet… still need to check the data…

avafinger @.***> schrieb am Mi. 4. Okt. 2023 um 15:24:

Did you attach the 2 sensors?

— Reply to this email directly, view it on GitHub https://github.com/initBasti/NanoPC-T4_armbian_configuration/issues/2#issuecomment-1746873498, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA3R4FEU26KWXH6TPSNJYLX5VPSVAVCNFSM4WCVSWDKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCNZUGY4DOMZUHE4A . You are receiving this because you commented.Message ID: @.***>

initBasti / NanoPC-T4_armbian_configuration

Add support for OV13850 DUAL camera #2