search

raspberrypi / linux

Kernel source tree for Raspberry Pi-provided kernel builds. Issues unrelated to the linux kernel should be posted on the community forum at https://forums.raspberrypi.com/
Other
11.19k stars 5.02k forks source link

libcamera: BCM2835_ISP_BASE different between 5.10.y and libcamera vs 5.12.y and 5.13.y #4440

Open olivierschonken opened 3 years ago

olivierschonken commented 3 years ago

Describe the bug Due to a mismatch in the BCM2835_ISP_BASE between libcamera and kernel 5.12.y and kernel 5.13.y, libcamera will not work with rpi unless its v4l2-controls.h is also modified to match the kernel v4l2-controls.h

kernel 5.12.y and 5.13.y v4l2-controls.h

define V4L2_CID_USER_BCM2835_ISP_BASE (V4L2_CID_USER_BASE + 0x10f0)

kernel 5.10.y and libcamera v4l2-controls.h

define V4L2_CID_USER_BCM2835_ISP_BASE (V4L2_CID_USER_BASE + 0x10e0)

To reproduce Use kernel 5.12.y or 5.13.y with current libcamera HEAD cam -c 1 -C 1

Expected behaviour root@raspberrypi4-64:~# cam -c 1 -C 1 [1:54:10.449586656] [622] INFO Camera camera_manager.cpp:296 libcamera v0.0.0+2747-fb970d9c-dirty (2021-07-08T15:00:29+00:00) [1:54:10.479843682] [623] WARN CameraSensor camera_sensor.cpp:197 'ov5647 10-0036': Recommended V4L2 control 0x009a0922 not supported [1:54:10.479959532] [623] WARN CameraSensor camera_sensor.cpp:249 'ov5647 10-0036': The sensor kernel driver needs to be fixed [1:54:10.480060234] [623] WARN CameraSensor camera_sensor.cpp:251 'ov5647 10-0036': See Documentation/sensor_driver_requirements.rst in the libcamera sources for more information [1:54:10.481761242] [623] WARN CameraSensor camera_sensor.cpp:403 'ov5647 10-0036': Failed to retrieve the camera location Using camera /base/soc/i2c0mux/i2c@1/ov5647@36 [1:54:10.757968853] [622] WARN V4L2 v4l2_pixelformat.cpp:181 Unsupported V4L2 pixel format XB24 [1:54:10.759028946] [622] INFO Camera camera.cpp:912 configuring streams: (0) 800x600-NV12 [1:54:10.759326886] [623] INFO RPI raspberrypi.cpp:122 Mode: 2592x1944 fmt BG10 Score: 2284 (best 2284) [1:54:10.759443328] [623] INFO RPI raspberrypi.cpp:122 Mode: 1920x1080 fmt BG10 Score: 2066.67 (best 2066.67) [1:54:10.759524012] [623] INFO RPI raspberrypi.cpp:122 Mode: 1296x972 fmt BG10 Score: 1717 (best 1717) [1:54:10.759600048] [623] INFO RPI raspberrypi.cpp:122 Mode: 640x480 fmt BG10 Score: 2060 (best 1717) [1:54:10.759678250] [623] INFO RPI raspberrypi.cpp:122 Mode: 2592x1944 fmt pBAA Score: 1784 (best 1717) [1:54:10.759753527] [623] INFO RPI raspberrypi.cpp:122 Mode: 1920x1080 fmt pBAA Score: 1566.67 (best 1566.67) [1:54:10.759829507] [623] INFO RPI raspberrypi.cpp:122 Mode: 1296x972 fmt pBAA Score: 1217 (best 1217) [1:54:10.759904172] [623] INFO RPI raspberrypi.cpp:122 Mode: 640x480 fmt pBAA Score: 1560 (best 1217) [1:54:10.760168797] [623] INFO RPI raspberrypi.cpp:619 Sensor: /base/soc/i2c0mux/i2c@1/ov5647@36 - Selected mode: 1296x972-pBAA [1:54:10.792911540] [623] INFO RPISTREAM rpi_stream.cpp:122 No buffers available for ISP Output0 [1:54:10.793035223] [623] INFO RPISTREAM rpi_stream.cpp:122 No buffers available for ISP Output0 [1:54:10.793097574] [623] INFO RPISTREAM rpi_stream.cpp:122 No buffers available for ISP Output0 [1:54:10.793153203] [623] INFO RPISTREAM rpi_stream.cpp:122 No buffers available for ISP Output0 Capture until user interrupts by SIGINT [1:54:10.896572493] [623] INFO RPI raspberrypi.cpp:1597 Dropping frame at the request of the IPA (7 left) [1:54:10.941121203] [623] INFO RPI raspberrypi.cpp:1597 Dropping frame at the request of the IPA (6 left) [1:54:10.975176128] [623] INFO RPI raspberrypi.cpp:1597 Dropping frame at the request of the IPA (5 left) [1:54:11.011654716] [623] INFO RPI raspberrypi.cpp:1597 Dropping frame at the request of the IPA (4 left) [1:54:11.041434583] [623] INFO RPI raspberrypi.cpp:1597 Dropping frame at the request of the IPA (3 left) [1:54:11.074707577] [623] INFO RPI raspberrypi.cpp:1597 Dropping frame at the request of the IPA (2 left) [1:54:11.141787814] [623] INFO RPI raspberrypi.cpp:1597 Dropping frame at the request of the IPA (1 left) [1:54:11.208628796] [623] INFO RPI raspberrypi.cpp:1597 Dropping frame at the request of the IPA (0 left) 6851.241004 (0.00 fps) stream0 seq: 000008 bytesused: 720000 6851.301087 (16.64 fps) stream0 seq: 000009 bytesused: 720000 6851.361170 (16.64 fps) stream0 seq: 000010 bytesused: 720000 6851.421255 (16.64 fps) stream0 seq: 000011 bytesused: 720000 6851.481339 (16.64 fps) stream0 seq: 000012 bytesused: 720000

Actual behaviour root@raspberrypi4-64:~# cam -c 1 -C 1 [0:00:50.949234727] [290] INFO Camera camera_manager.cpp:294 libcamera v0.0.0+2582-2a103ba1-dirty (2021-07-04T00:48:24+00:00) [0:00:50.980585042] [291] WARN CameraSensor camera_sensor.cpp:307 'ov5647 10-0036': Recommended V4L2 control 0x009a0922 not supported [0:00:50.981056579] [291] WARN CameraSensor camera_sensor.cpp:359 'ov5647 10-0036': The sensor kernel driver needs to be fixed [0:00:50.981136653] [291] WARN CameraSensor camera_sensor.cpp:361 'ov5647 10-0036': See Documentation/sensor_driver_requirements.rst in the libcamera sources for more information [0:00:50.983345857] [291] WARN CameraSensorProperties camera_sensor_properties.cpp:61 No static properties available for 'ov5647' [0:00:50.983423968] [291] WARN CameraSensorProperties camera_sensor_properties.cpp:63 Please consider updating the camera sensor properties database [0:00:50.983959857] [291] WARN CameraSensor camera_sensor.cpp:487 'ov5647 10-0036': Failed to retrieve the camera location Using camera /base/soc/i2c0mux/i2c@1/ov5647@36 [0:00:51.030133450] [290] WARN V4L2 v4l2_pixelformat.cpp:180 Unsupported V4L2 pixel format XB24 [0:00:51.031235820] [290] INFO Camera camera.cpp:905 configuring streams: (0) 800x600-NV12 [0:00:51.031607542] [291] INFO RPI raspberrypi.cpp:122 Mode: 2592x1944 fmt BG10 Score: 2284 (best 2284) [0:00:51.031724042] [291] INFO RPI raspberrypi.cpp:122 Mode: 1920x1080 fmt BG10 Score: 2066.67 (best 2066.67) [0:00:51.031807802] [291] INFO RPI raspberrypi.cpp:122 Mode: 1296x972 fmt BG10 Score: 1717 (best 1717) [0:00:51.031886098] [291] INFO RPI raspberrypi.cpp:122 Mode: 640x480 fmt BG10 Score: 2060 (best 1717) [0:00:51.031995487] [291] INFO RPI raspberrypi.cpp:122 Mode: 2592x1944 fmt pBAA Score: 1784 (best 1717) [0:00:51.032133227] [291] INFO RPI raspberrypi.cpp:122 Mode: 1920x1080 fmt pBAA Score: 1566.67 (best 1566.67) [0:00:51.032242042] [291] INFO RPI raspberrypi.cpp:122 Mode: 1296x972 fmt pBAA Score: 1217 (best 1217) [0:00:51.032346283] [291] INFO RPI raspberrypi.cpp:122 Mode: 640x480 fmt pBAA Score: 1560 (best 1217) [0:00:51.032645968] [291] INFO RPI raspberrypi.cpp:619 Sensor: /base/soc/i2c0mux/i2c@1/ov5647@36 - Selected mode: 1296x972-pBAA [0:00:51.034692653] [291] ERROR IPARPI raspberrypi.cpp:549 Unable to find ISP control 0x009819e1 [0:00:51.034778598] [291] ERROR IPARPI raspberrypi.cpp:347 ISP control validation failed. [0:00:51.034837042] [291] ERROR RPI raspberrypi.cpp:1291 IPA configuration failed! [0:00:51.034904802] [291] ERROR RPI raspberrypi.cpp:756 Failed to configure the IPA: -32 Failed to configure camera

6by9 commented 3 years ago

@naushir @davidplowman This is going to be ongoing entertainment until bcm2835-isp gets mainlined. V4L2_CID_USER_CODA_BASE has taken (V4L2_CID_USER_BASE + 0x10e0), and V4L2_CID_USER_CCS_BASE has taken (V4L2_CID_USER_BASE + 0x10f0) (which is a duplicate definition with V4L2_CID_USER_BCM2835_ISP_BASE.

We could go for V4L2_CID_USER_BASE + 0xfff0 to take the very top allocation and avoid issues until it is merged. Or we keep it at V4L2_CID_USER_BASE + 0x10e0 and ignore the collision with CODA (which only uses 1 control as V4L2_CID_CODA_MB_ERR_CNT, and doesn't add any of the magic flags in the core v4l2 during creation). This option sounds the easiest.

naushir commented 3 years ago

Sigh, this is indeed a pain. My vote is to keep it at V4L2_CID_USER_BASE + 0x10e0 just so that we do not have to sync base changes with the uapi header in libcamera.

6by9 commented 2 years ago

4806 fixes this for 5.15. Sorry it's taken a while for the PR to actually get created.

flatsiedatsie commented 2 years ago

I found this Github issue while looking at an error using picamera2 on a Raspberry Pi Zero 2W using the latest Raspbian Lite 32 bit (bullseye). Source

kernel: ov5647 10-0036: Failed to program sensor mode: -121
kernel: ov5647 10-0036: stream start failed: -121
kernel: unicam 3f801000.csi: stream on failed in subdev

Which seems to originate here: https://github.com/raspberrypi/linux/blob/rpi-5.15.y/drivers/media/i2c/ov5647.c

This is the context:

/boot/config.txt:
dtparam=i2c_arm=on
dtparam=spi=on
dtparam=audio=on
camera_auto_detect=1
display_auto_detect=1
dtoverlay=vc4-fkms-v3d
max_framebuffers=2
disable_overscan=1
[cm4]
otg_mode=1
[all]
[pi4]
arm_boost=1
[all]
initramfs initrd followkernel
ramfsfile=initrd
ramfsaddr=-1
dtoverlay=i2s-mmap
dtparam=i2s=on
disable_splash=1
max_usb_current=1

/boot/cmdline.txt:
console=tty3 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait consoleblank=0 net.ifnames=0 quiet plymouth.ignore-serial-consoles splash logo.nologo vt.global_cursor_default=0 init=/bin/ro-root.sh

/lib/modules (kernel)
5.15.61+
5.15.61-v7+
5.15.61-v7l+
5.15.61-v8+

Libcamera:
Available cameras
-----------------
0 : ov5647 [2592x1944] (/base/soc/i2c0mux/i2c@1/ov5647@36)
    Modes: 'SGBRG10_CSI2P' : 640x480 [30.00 fps - (0, 0)/0x0 crop]
                             1296x972 [30.00 fps - (0, 0)/0x0 crop]
                             1920x1080 [30.00 fps - (0, 0)/0x0 crop]
                             2592x1944 [30.00 fps - (0, 0)/0x0 crop]

--------------------------------------------- lsmod

Module                  Size  Used by
cmac                   16384  35
algif_hash             16384  17
aes_arm_bs             24576  34
crypto_simd            16384  1 aes_arm_bs
cryptd                 24576  18 crypto_simd
algif_skcipher         16384  17
af_alg                 28672  70 algif_hash,algif_skcipher
bnep                   20480  2
hci_uart               40960  1
btbcm                  20480  1 hci_uart
xt_state               16384  0
xt_conntrack           16384  2
nft_chain_nat          16384  1
xt_REDIRECT            16384  2
nf_nat                 49152  2 nft_chain_nat,xt_REDIRECT
nf_conntrack          139264  4 xt_state,xt_conntrack,xt_REDIRECT,nf_nat
snd_soc_simple_card    20480  0
snd_soc_simple_card_utils    24576  1 snd_soc_simple_card
nf_defrag_ipv6         20480  1 nf_conntrack
nf_defrag_ipv4         16384  1 nf_conntrack
hidp                   24576  0
bluetooth             409600  29 hidp,hci_uart,bnep,btbcm
nft_counter            16384  6
ecdh_generic           16384  2 bluetooth
ecc                    40960  1 ecdh_generic
xt_mark                16384  4
xt_tcpudp              16384  6
nft_compat             20480  14
nf_tables             212992  17 nft_compat,nft_chain_nat,nft_counter
nfnetlink              20480  2 nft_compat,nf_tables
8021q                  32768  0
garp                   16384  1 8021q
stp                    16384  1 garp
llc                    16384  2 garp,stp
ov5647                 20480  2
spidev                 20480  0
brcmfmac              335872  0
vc4                   290816  0
brcmutil               20480  1 brcmfmac
sha256_generic         16384  0
cec                    49152  1 vc4
cfg80211              765952  1 brcmfmac
drm_kms_helper        274432  3 vc4
syscopyarea            16384  1 drm_kms_helper
sysfillrect            16384  1 drm_kms_helper
sysimgblt              16384  1 drm_kms_helper
fb_sys_fops            16384  1 drm_kms_helper
raspberrypi_hwmon      16384  0
i2c_mux_pinctrl        16384  0
i2c_mux                16384  1 i2c_mux_pinctrl
bcm2835_unicam         45056  3
rfkill                 32768  6 bluetooth,cfg80211
bcm2835_v4l2           45056  0
bcm2835_codec          45056  0
bcm2835_isp            32768  5
v4l2_dv_timings        36864  1 bcm2835_unicam
v4l2_mem2mem           36864  1 bcm2835_codec
v4l2_fwnode            24576  2 bcm2835_unicam,ov5647
bcm2835_mmal_vchiq     32768  3 bcm2835_isp,bcm2835_codec,bcm2835_v4l2
v4l2_async             24576  3 bcm2835_unicam,v4l2_fwnode,ov5647
videobuf2_vmalloc      16384  1 bcm2835_v4l2
videobuf2_dma_contig    20480  16 bcm2835_unicam,bcm2835_isp,bcm2835_codec
videobuf2_memops       16384  2 videobuf2_dma_contig,videobuf2_vmalloc
videobuf2_v4l2         32768  5 bcm2835_unicam,bcm2835_isp,bcm2835_codec,bcm2835_v4l2,v4l2_mem2mem
videobuf2_common       65536  9 bcm2835_unicam,bcm2835_isp,bcm2835_codec,videobuf2_dma_contig,videobuf2_vmalloc,videobuf2_memops,bcm2835_v4l2,v4l2_mem2mem,videobuf2_v4l2
videodev              245760  15 bcm2835_unicam,bcm2835_isp,ov5647,bcm2835_codec,videobuf2_common,bcm2835_v4l2,v4l2_mem2mem,videobuf2_v4l2,v4l2_async
i2c_bcm2835            16384  0
spi_bcm2835            20480  0
snd_soc_bcm2835_i2s    16384  2
mc                     45056  11 bcm2835_unicam,bcm2835_isp,ov5647,bcm2835_codec,videobuf2_common,videodev,v4l2_mem2mem,videobuf2_v4l2,v4l2_async
snd_bcm2835            24576  0
vc_sm_cma              32768  2 bcm2835_isp,bcm2835_mmal_vchiq
uio_pdrv_genirq        16384  0
fixed                  16384  3
uio                    20480  1 uio_pdrv_genirq
snd_soc_wm8960         36864  1
snd_soc_core          233472  5 vc4,snd_soc_simple_card_utils,snd_soc_bcm2835_i2s,snd_soc_wm8960,snd_soc_simple_card
snd_compress           20480  1 snd_soc_core
snd_pcm_dmaengine      16384  1 snd_soc_core
snd_pcm               114688  6 snd_compress,snd_pcm_dmaengine,snd_soc_bcm2835_i2s,snd_bcm2835,snd_soc_core,snd_soc_wm8960
snd_timer              32768  1 snd_pcm
snd                    81920  6 snd_compress,snd_timer,snd_bcm2835,snd_soc_core,snd_soc_wm8960,snd_pcm
drm                   536576  3 vc4,drm_kms_helper
regmap_i2c             16384  1 snd_soc_wm8960
i2c_dev                20480  0
fuse                  131072  1
drm_panel_orientation_quirks    16384  1 drm
backlight              20480  2 drm_kms_helper,drm
ip_tables              28672  0
x_tables               32768  7 xt_state,ip_tables,nft_compat,xt_mark,xt_tcpudp,xt_conntrack,xt_REDIRECT
ipv6                  524288  56
overlay               114688  1

I hope this is of use.

flatsiedatsie commented 2 years ago

Some more details:

2022-09-06 11:34:23.747 ERROR  : candlecam: [0:01:12.793085464] [1277]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0+3866-0c55e522
2022-09-06 11:34:23.859 ERROR  : candlecam: [0:01:12.920192250] [1286]  INFO RPI raspberrypi.cpp:1374 Registered camera /base/soc/i2c0mux/i2c@1/ov5647@36 to Unicam device /dev/media3 and ISP device /dev/media0
2022-09-06 11:34:23.901 INFO   : candlecam: self.picam.camera_controls: ['__call__', '__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__name__', '__ne__', '__new__', '__qualname__', '__reduce__', '__reduce_ex__', '__repr__', '__self__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__text_signature__']
2022-09-06 11:34:23.906 INFO   : candlecam: intial_picam_config : {'use_case': 'video', 'transform': <libcamera.Transform 'identity'>, 'colour_space': <libcamera.ColorSpace 'sYCC'>, 'buffer_count': 6, 'main': {'format': 'XBGR8888', 'size': (640, 480)}, 'lores': None, 'raw': None, 'controls': {'NoiseReductionMode': <NoiseReductionModeEnum.Fast: 1>, 'FrameDurationLimits': (33333, 33333)}, 'display': 'main', 'encode': 'main'}
2022-09-06 11:34:23.933 ERROR  : candlecam: [0:01:12.965003560] [1277]  INFO Camera camera.cpp:1035 configuring streams: (0) 640x480-XBGR8888
2022-09-06 11:34:23.936 ERROR  : candlecam: [0:01:12.966647440] [1286]  INFO RPI raspberrypi.cpp:761 Sensor: /base/soc/i2c0mux/i2c@1/ov5647@36 - Selected sensor format: 640x480-SGBRG10_1X10 - Selected unicam format: 640x480-pGAA
022-09-06 11:34:25.861 ERROR  : candlecam: [0:01:13.303029644] [1286] ERROR V4L2 v4l2_videodevice.cpp:1890 /dev/video0[17:cap]: Failed to start streaming: Remote I/O error
2022-09-06 11:34:25.863 ERROR  : candlecam: 2022-09-06T09:34:24.234Z | ERROR    | Camera did not start properly.
2022-09-06 11:34:25.875 ERROR  : candlecam: Exception in thread Thread-2:
2022-09-06 11:34:25.909 ERROR  : candlecam: Traceback (most recent call last):
2022-09-06 11:34:25.915 ERROR  : candlecam:   File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2022-09-06 11:34:25.919 ERROR  : candlecam:     self.run()
2022-09-06 11:34:25.924 ERROR  : candlecam:   File "/usr/lib/python3.9/threading.py", line 892, in run
2022-09-06 11:34:25.929 ERROR  : candlecam:     self._target(*self._args, **self._kwargs)
2022-09-06 11:34:25.932 ERROR  : candlecam:   File "/home/pi/.webthings/addons/candlecam/pkg/candlecam.py", line 939, in run_picam
2022-09-06 11:34:25.936 ERROR  : candlecam:     self.picam.start()
2022-09-06 11:34:25.995 ERROR  : candlecam:   File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 872, in start
2022-09-06 11:34:26.007 ERROR  : candlecam:     self.start_()
2022-09-06 11:34:27.644 ERROR  : candlecam:   File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 844, in start_
2022-09-06 11:34:27.646 ERROR  : candlecam:     raise RuntimeError("Camera did not start properly.")
2022-09-06 11:34:27.648 ERROR  : candlecam: RuntimeError: Camera did not start properly.
flatsiedatsie commented 2 years ago

Ran the same SD card in an Pi 3B with a different camera module, and it worked OK. Might just be the camera module.

// yes, definitely a hardware issue, sorry. Switched module AND cable, and now it works on the Pi Zero 2.