ayufan / camera-streamer

High-performance low-latency camera streamer for Raspberry PI's
291 stars 46 forks source link

Unable to get any pixel from imx219/Pi 4b/Bookworm #138

Closed arcadien closed 3 weeks ago

arcadien commented 4 months ago

Hello, I post my problem here since i'm a bit lost into all components involved in camera streaming on the Raspberry platform. Here is my setup:

After long readings on the internets, I succeeded streaming the C615 using the camera-streamer backend in webrtc mode. As far as i understand, it means hardware acceleration is available and activated for that device. My problem comes from the Raspberry cam. It is detected so that hardware should be ok:

15:27 $ libcamera-hello  --list-camera
Available cameras
-----------------
0 : imx219 [3280x2464 10-bit RGGB] (/base/soc/i2c0mux/i2c@1/imx219@10)
    Modes: 'SRGGB10_CSI2P' : 640x480 [206.65 fps - (1000, 752)/1280x960 crop]
                             1640x1232 [41.85 fps - (0, 0)/3280x2464 crop]
                             1920x1080 [47.57 fps - (680, 692)/1920x1080 crop]
                             3280x2464 [21.19 fps - (0, 0)/3280x2464 crop]
           'SRGGB8' : 640x480 [206.65 fps - (1000, 752)/1280x960 crop]
                      1640x1232 [83.70 fps - (0, 0)/3280x2464 crop]
                      1920x1080 [47.57 fps - (680, 692)/1920x1080 crop]
                      3280x2464 [21.19 fps - (0, 0)/3280x2464 crop]

By the way, streaming with Crowsnest does not work. The command line it "produces" is: 15:30 $ /home/pi/crowsnest/bin/camera-streamer/camera-streamer --http-port=8080 --camera-path=/base/soc/i2c0mux/i2c@1/imx219@10 --camera-type=libcamera --camera-format=YUYV --camera-snapshot.height=1080 --camera-width=1920 --camera-height=1080 --camera-fps=30 --camera-auto_reconnect=1

When I launch that command on the terminal, i get this error:

[5:04:42.729452782] [15275] ERROR V4L2 v4l2_videodevice.cpp:1697 /dev/video13[24:out]: Failed to queue buffer 0: Invalid argument
[5:04:42.729539281] [15275] ERROR RPISTREAM rpi_stream.cpp:276 Failed to queue buffer for ISP Input

I spend hours reading various interesting posts about camera models, old and new (libcamera) implementations, CMA memory, start/fixup files and so on. I tried a lot of parameter combination, but never succeeded in having any output.

Ustreamer command line and log

$ /home/pi/crowsnest/bin/ustreamer/ustreamer --host 127.0.0.1 -p 8080 -d /dev/video2 --device-timeout=2 -r 1920x1080 -f 30 --allow-origin=* --static /home/pi/crowsnest/ustreamer-www --debug

================================================================================
-- INFO  [18953.449          ] -- Device fd=8 opened
-- DEBUG [18953.449          ] -- Querying device capabilities ...
-- INFO  [18953.449          ] -- Using capture type: single-planar
-- INFO  [18953.449          ] -- Using input channel: 0
-- DEBUG [18953.449          ] -- Using TV standard: DEFAULT
-- DEBUG [18953.449          ] -- Probing device format=YUYV, stride=3840, resolution=1920x1080 ...
-- INFO  [18953.449          ] -- Using resolution: 1920x1080
-- INFO  [18953.449          ] -- Using format: YUYV
-- DEBUG [18953.449          ] -- Querying HW FPS ...
-- INFO  [18953.449          ] -- Querying HW FPS changing is not supported
-- INFO  [18953.449          ] -- Using IO method: MMAP
-- DEBUG [18953.449          ] -- Requesting 5 device buffers for MMAP ...
-- DEBUG [18953.451          ] -- HTTP: Updating exposed frame (online=0) ...
-- VERB  [18953.451          ] -- HTTP: Exposed frame: online=0, exp_time=0.000000
-- INFO  [18953.472          ] -- Requested 5 device buffers, got 5
-- DEBUG [18953.472          ] -- Allocating device buffers ...
-- DEBUG [18953.472          ] -- Calling us_xioctl(VIDIOC_QUERYBUF) for device buffer=0 ...
-- DEBUG [18953.472          ] -- Mapping device buffer=0 ...
-- DEBUG [18953.472          ] -- Calling us_xioctl(VIDIOC_QUERYBUF) for device buffer=1 ...
-- DEBUG [18953.472          ] -- Mapping device buffer=1 ...
-- DEBUG [18953.472          ] -- Calling us_xioctl(VIDIOC_QUERYBUF) for device buffer=2 ...
-- DEBUG [18953.472          ] -- Mapping device buffer=2 ...
-- DEBUG [18953.472          ] -- Calling us_xioctl(VIDIOC_QUERYBUF) for device buffer=3 ...
-- DEBUG [18953.472          ] -- Mapping device buffer=3 ...
-- DEBUG [18953.472          ] -- Calling us_xioctl(VIDIOC_QUERYBUF) for device buffer=4 ...
-- DEBUG [18953.472          ] -- Mapping device buffer=4 ...
-- DEBUG [18953.472          ] -- Calling us_xioctl(VIDIOC_QBUF) for buffer=0 ...
-- DEBUG [18953.472          ] -- Calling us_xioctl(VIDIOC_QBUF) for buffer=1 ...
-- DEBUG [18953.472          ] -- Calling us_xioctl(VIDIOC_QBUF) for buffer=2 ...
-- DEBUG [18953.472          ] -- Calling us_xioctl(VIDIOC_QBUF) for buffer=3 ...
-- DEBUG [18953.472          ] -- Calling us_xioctl(VIDIOC_QBUF) for buffer=4 ...
-- DEBUG [18953.472          ] -- Device fd=8 initialized
-- DEBUG [18953.473          ] -- Starting device capturing ...
-- ERROR [18953.473          ] -- Can't start capturing: Invalid argument
-- DEBUG [18953.473          ] -- Releasing device buffers ...
-- DEBUG [18953.473          ] -- Closing device ...
-- INFO  [18953.477          ] -- Device fd=8 closed
-- INFO  [18953.477          ] -- Sleeping 1 seconds before new stream init ...
^C-- INFO  [18954.107          ] -- ===== Stopping by SIGINT =====
-- INFO  [18954.107          ] -- HTTP eventloop stopped
-- INFO  [18954.477          ] -- Bye-bye

Current config.txt

# For more options and information see
# http://rptl.io/configtxt
# Some settings may impact device functionality. See link above for details

# Uncomment some or all of these to enable the optional hardware interfaces
dtparam=i2c_arm=on
#dtparam=i2s=on
dtparam=spi=on

# Enable audio (loads snd_bcm2835)
# dtparam=audio=on

# Additional overlays and parameters are documented
# /boot/firmware/overlays/README

# Automatically load overlays for detected DSI displays
display_auto_detect=1

# Automatically load initramfs files, if found
auto_initramfs=1

# Don't have the firmware create an initial video= setting in cmdline.txt.
# Use the kernel's default instead.
disable_fw_kms_setup=1

# Run in 64-bit mode
arm_64bit=1

# Disable compensation for displays with overscan
disable_overscan=1

# Run as fast as firmware / board allows
arm_boost=1

[cm4]
# Enable host mode on the 2711 built-in XHCI USB controller.
# This line should be removed if the legacy DWC2 controller is required
# (e.g. for USB device mode) or if USB support is not required.
otg_mode=1

[all]

# Enable DRM VC4 V3D driver
dtoverlay=vc4-kms-v3d
max_framebuffers=4

# DS18B20
dtoverlay=w1-gpio

# legacy video stack
#start_x=1

start_file=start4x.elf
fixup_file=fixup4x.dat

# Automatically load overlays for detected cameras
camera_auto_detect=1
gpu_mem=128

Complete camera-streamer output

15:31 $ /home/pi/crowsnest/bin/camera-streamer/camera-streamer --http-port=8080 --camera-path=/base/soc/i2c0mux/i2c@1/imx219@10 --camera-type=libcamera --camera-format=YUYV --camera-snapshot.height=1080 --camera-width=1920 --camera-height=1080 --camera-fps=30 --camera-auto_reconnect=1
/home/pi/crowsnest/bin/camera-streamer/camera-streamer Version:  (54bdd9d)
util/http/http.c: ?: HTTP listening on 127.0.0.1:8080.
[5:04:29.040372046] [15262] ERROR IPAModule ipa_module.cpp:172 Symbol ipaModuleInfo not found
[5:04:29.040490193] [15262] ERROR IPAModule ipa_module.cpp:292 v4l2-compat.so: IPA module has no valid info
[5:04:29.040617118] [15262]  INFO Camera camera_manager.cpp:284 libcamera v0.2.0+46-075b54d5
[5:04:29.088188244] [15275]  WARN RPiSdn sdn.cpp:39 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise
[5:04:29.090142892] [15275]  WARN RPI vc4.cpp:393 Mismatch between Unicam and CamHelper for embedded data usage!
[5:04:29.090841312] [15275]  INFO RPI vc4.cpp:447 Registered camera /base/soc/i2c0mux/i2c@1/imx219@10 to Unicam device /dev/media5 and ISP device /dev/media1
device/libcamera/device.cc: CAMERA: Device path=/base/soc/i2c0mux/i2c@1/imx219@10 opened
[5:04:29.092772849] [15262]  INFO Camera camera.cpp:1183 configuring streams: (0) 1920x1080-YUYV
[5:04:29.093201178] [15275]  INFO RPI vc4.cpp:611 Sensor: /base/soc/i2c0mux/i2c@1/imx219@10 - Selected sensor format: 1920x1080-SBGGR10_1X10 - Selected unicam format: 1920x1080-pBAA
[5:04:29.096389500] [15262]  INFO Camera camera.cpp:1183 configuring streams: (0) 1920x1080-YUYV (1) 1920x1080-SBGGR10_CSI2P
[5:04:29.096789015] [15275]  INFO RPI vc4.cpp:611 Sensor: /base/soc/i2c0mux/i2c@1/imx219@10 - Selected sensor format: 1920x1080-SBGGR10_1X10 - Selected unicam format: 1920x1080-pBAA
device/buffer_list.c: CAMERA:capture: Using: 1920x1080/YUYV, buffers=3, bytesperline=3840, sizeimage=0.0MiB
device/buffer_list.c: CAMERA:capture: Opened 3 buffers. Memory used: 11.9 MiB
device/buffer_list.c: CAMERA:capture:1: Using: 1920x1080/BG10, buffers=3, bytesperline=4128, sizeimage=0.0MiB
device/buffer_list.c: CAMERA:capture:1: Opened 3 buffers. Memory used: 12.8 MiB
device/v4l2/device.c: SNAPSHOT: Device path=/dev/video31 fd=41 opened
device/v4l2/buffer_list.c: SNAPSHOT:output:mplane: Requested resolution=1920x1080 is unavailable. Got 1920x1088.
device/buffer_list.c: SNAPSHOT:output:mplane: Using: 1920x1056/YUYV, buffers=3, bytesperline=3840, sizeimage=3.9MiB
device/buffer_list.c: SNAPSHOT:output:mplane: Opened 3 buffers. Memory used: 0.0 MiB
device/buffer_list.c: SNAPSHOT:capture:mplane: Using: 1920x1056/JPEG, buffers=3, bytesperline=0, sizeimage=4.0MiB
device/buffer_list.c: SNAPSHOT:capture:mplane: Opened 3 buffers. Memory used: 12.0 MiB
device/v4l2/device.c: VIDEO: Device path=/dev/video11 fd=45 opened
device/buffer_list.c: VIDEO:output:mplane: Using: 1920x1080/YUYV, buffers=3, bytesperline=3840, sizeimage=4.0MiB
device/buffer_list.c: VIDEO:output:mplane: Opened 3 buffers. Memory used: 0.0 MiB
device/buffer_list.c: VIDEO:capture:mplane: Using: 1920x1080/H264, buffers=3, bytesperline=0, sizeimage=0.8MiB
device/buffer_list.c: VIDEO:capture:mplane: Opened 3 buffers. Memory used: 2.2 MiB
device/device.c: CAMERA: Setting frame interval_us=0 for FPS=30
device/buffer_list.c: CAMERA:capture: Streaming started... Was 0 of 3 enqueud
device/v4l2/device_options.c: SNAPSHOT: Configuring option 'Compression Quality' (009d0903) = 80
device/v4l2/device_options.c: VIDEO: Configuring option 'Repeat Sequence Header' (009909e2) = 1
device/v4l2/device_options.c: VIDEO: Configuring option 'Video Bitrate Mode' (009909ce) = 0
device/v4l2/device_options.c: VIDEO: Configuring option 'Video Bitrate' (009909cf) = 2000000
device/v4l2/device_options.c: VIDEO: Configuring option 'Repeat Sequence Header' (009909e2) = 5000000
device/v4l2/device_options.c: VIDEO: Configuring option 'H264 I-Frame Period' (00990a66) = 30
device/v4l2/device_options.c: VIDEO: Configuring option 'H264 Level' (00990a67) = 11
device/v4l2/device_options.c: VIDEO: Configuring option 'H264 Profile' (00990a6b) = 4
device/v4l2/device_options.c: VIDEO: Configuring option 'H264 Minimum QP Value' (00990a61) = 16
device/v4l2/device_options.c: VIDEO: Configuring option 'H264 Maximum QP Value' (00990a62) = 32
device/links.c: ?: Link 0: CAMERA:capture[1920x1080/YUYV/3] => [SNAPSHOT:output:mplane[1920x1056/YUYV/3], VIDEO:output:mplane[1920x1080/YUYV/3]]
device/links.c: ?: Link 1: SNAPSHOT:capture:mplane[1920x1056/JPEG/3] => [SNAPSHOT-CAPTURE, STREAM-CAPTURE]
device/links.c: ?: Link 2: VIDEO:capture:mplane[1920x1080/H264/3] => [VIDEO-CAPTURE]
device/buffer_list.c: SNAPSHOT:output:mplane: Streaming started... Was 0 of 3 enqueud
device/buffer_list.c: VIDEO:output:mplane: Streaming started... Was 0 of 3 enqueud
device/buffer_list.c: SNAPSHOT:capture:mplane: Streaming started... Was 0 of 3 enqueud
device/buffer_list.c: VIDEO:capture:mplane: Streaming started... Was 0 of 3 enqueud
util/http/http.c: HTTP8080/0: Client connected 127.0.0.1 (fd=4).
util/http/http.c: HTTP8080/0: Request 'GET' '/' 'action=snapshot&bypassCache=1708785111260'
[5:04:42.729452782] [15275] ERROR V4L2 v4l2_videodevice.cpp:1697 /dev/video13[24:out]: Failed to queue buffer 0: Invalid argument
[5:04:42.729539281] [15275] ERROR RPISTREAM rpi_stream.cpp:276 Failed to queue buffer for ISP Input
Riconec commented 4 months ago

I had exactly the same issue but on raspbian bookworm on CM4 with raspi camera v1 (I was told this is a clone). Libcamera was detecting it correctly, libcamera-jpeg produced photos but camera streamer was giving server errors and no picture

зображення

libcamera-hello --list-cameras
Available cameras
-----------------
0 : ov5647 [2592x1944 10-bit GBRG] (/base/soc/i2c0mux/i2c@1/ov5647@36)
    Modes: 'SGBRG10_CSI2P' : 640x480 [30.00 fps - (0, 0)/0x0 crop]
                             1296x972 [30.00 fps - (0, 0)/0x0 crop]
                             1920x1080 [30.00 fps - (0, 0)/0x0 crop]
                             2592x1944 [30.00 fps - (0, 0)/0x0 crop]
~/crowsnest/bin/camera-streamer/camera-streamer --http-port=8080 --camera-path=/base/soc/i2c0mux/i2c@1/ov5647@36 --camera-type=libcamera --camera-format=YUYV --camera-snapshot.height=1080 --camera-width=640 --camera-height=480 --camera-fps=15 --camera-auto_reconnect=1
/home/rico/crowsnest/bin/camera-streamer/camera-streamer Version:  (54bdd9d)
util/http/http.c: ?: HTTP listening on 127.0.0.1:8080.
[0:36:59.893826082] [47498]  INFO Camera camera_manager.cpp:284 libcamera v0.2.0+46-075b54d5
[0:36:59.939146557] [47515]  WARN RPiSdn sdn.cpp:39 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise
[0:36:59.942456905] [47515]  INFO RPI vc4.cpp:447 Registered camera /base/soc/i2c0mux/i2c@1/ov5647@36 to Unicam device /dev/media4 and ISP device /dev/media0
device/libcamera/device.cc: CAMERA: Device path=/base/soc/i2c0mux/i2c@1/ov5647@36 opened
[0:36:59.943386282] [47498]  INFO Camera camera.cpp:1183 configuring streams: (0) 640x480-YUYV
[0:36:59.945713939] [47515]  INFO RPI vc4.cpp:611 Sensor: /base/soc/i2c0mux/i2c@1/ov5647@36 - Selected sensor format: 640x480-SGBRG10_1X10 - Selected unicam format: 640x480-pGAA
[0:36:59.947969670] [47498]  INFO Camera camera.cpp:1183 configuring streams: (0) 640x480-YUYV (1) 640x480-SGBRG10_CSI2P
[0:36:59.948494645] [47515]  INFO RPI vc4.cpp:611 Sensor: /base/soc/i2c0mux/i2c@1/ov5647@36 - Selected sensor format: 640x480-SGBRG10_1X10 - Selected unicam format: 640x480-pGAA
device/buffer_list.c: CAMERA:capture: Using: 640x480/YUYV, buffers=3, bytesperline=1280, sizeimage=0.0MiB
device/buffer_list.c: CAMERA:capture: Opened 3 buffers. Memory used: 1.8 MiB
device/buffer_list.c: CAMERA:capture:1: Using: 640x480/GB10, buffers=3, bytesperline=3264, sizeimage=0.0MiB
device/buffer_list.c: CAMERA:capture:1: Opened 3 buffers. Memory used: 4.5 MiB
device/v4l2/device.c: SNAPSHOT: Device path=/dev/video31 fd=39 opened
device/buffer_list.c: SNAPSHOT:output:mplane: Using: 640x480/YUYV, buffers=3, bytesperline=1280, sizeimage=0.6MiB
device/buffer_list.c: SNAPSHOT:output:mplane: Opened 3 buffers. Memory used: 0.0 MiB
device/buffer_list.c: SNAPSHOT:capture:mplane: Using: 640x480/JPEG, buffers=3, bytesperline=0, sizeimage=4.0MiB
device/buffer_list.c: SNAPSHOT:capture:mplane: Opened 3 buffers. Memory used: 12.0 MiB
device/v4l2/device.c: VIDEO: Device path=/dev/video11 fd=43 opened
device/buffer_list.c: VIDEO:output:mplane: Using: 640x480/YUYV, buffers=3, bytesperline=1280, sizeimage=0.6MiB
device/buffer_list.c: VIDEO:output:mplane: Opened 3 buffers. Memory used: 0.0 MiB
device/buffer_list.c: VIDEO:capture:mplane: Using: 640x480/H264, buffers=3, bytesperline=0, sizeimage=0.5MiB
device/buffer_list.c: VIDEO:capture:mplane: Opened 3 buffers. Memory used: 1.5 MiB
device/device.c: CAMERA: Setting frame interval_us=0 for FPS=15
device/buffer_list.c: CAMERA:capture: Streaming started... Was 0 of 3 enqueud
device/v4l2/device_options.c: SNAPSHOT: Configuring option 'Compression Quality' (009d0903) = 80
device/v4l2/device_options.c: VIDEO: Configuring option 'Repeat Sequence Header' (009909e2) = 1
device/v4l2/device_options.c: VIDEO: Configuring option 'Video Bitrate Mode' (009909ce) = 0
device/v4l2/device_options.c: VIDEO: Configuring option 'Video Bitrate' (009909cf) = 2000000
device/v4l2/device_options.c: VIDEO: Configuring option 'Repeat Sequence Header' (009909e2) = 5000000
device/v4l2/device_options.c: VIDEO: Configuring option 'H264 I-Frame Period' (00990a66) = 30
device/v4l2/device_options.c: VIDEO: Configuring option 'H264 Level' (00990a67) = 11
device/v4l2/device_options.c: VIDEO: Configuring option 'H264 Profile' (00990a6b) = 4
device/v4l2/device_options.c: VIDEO: Configuring option 'H264 Minimum QP Value' (00990a61) = 16
device/v4l2/device_options.c: VIDEO: Configuring option 'H264 Maximum QP Value' (00990a62) = 32
device/links.c: ?: Link 0: CAMERA:capture[640x480/YUYV/3] => [SNAPSHOT:output:mplane[640x480/YUYV/3], VIDEO:output:mplane[640x480/YUYV/3]]
device/links.c: ?: Link 1: SNAPSHOT:capture:mplane[640x480/JPEG/3] => [SNAPSHOT-CAPTURE, STREAM-CAPTURE]
device/links.c: ?: Link 2: VIDEO:capture:mplane[640x480/H264/3] => [VIDEO-CAPTURE]
device/buffer_list.c: SNAPSHOT:output:mplane: Streaming started... Was 0 of 3 enqueud
device/buffer_list.c: VIDEO:output:mplane: Streaming started... Was 0 of 3 enqueud
device/buffer_list.c: SNAPSHOT:capture:mplane: Streaming started... Was 0 of 3 enqueud
device/buffer_list.c: VIDEO:capture:mplane: Streaming started... Was 0 of 3 enqueud
util/http/http.c: HTTP8080/0: Client connected 127.0.0.1 (fd=4).
cat /boot/firmware/config.txt
# For more options and information see
# http://rptl.io/configtxt
# Some settings may impact device functionality. See link above for details

# Uncomment some or all of these to enable the optional hardware interfaces
dtparam=i2c_arm=on
#dtparam=i2s=on
dtparam=spi=on

# Enable audio (loads snd_bcm2835)
dtparam=audio=on

# Additional overlays and parameters are documented
# /boot/firmware/overlays/README

# Automatically load overlays for detected cameras
camera_auto_detect=0
dtoverlay=ov5647
#start_x=1
gpu_mem=256

# Automatically load overlays for detected DSI displays
display_auto_detect=1

# Automatically load initramfs files, if found
auto_initramfs=1

# Enable DRM VC4 V3D driver
dtoverlay=vc4-kms-v3d
max_framebuffers=2

# Don't have the firmware create an initial video= setting in cmdline.txt.
# Use the kernel's default instead.
disable_fw_kms_setup=1

# Run in 64-bit mode
arm_64bit=1

# Disable compensation for displays with overscan
disable_overscan=1

# Run as fast as firmware / board allows
arm_boost=1

[cm4]
# Enable host mode on the 2711 built-in XHCI USB controller.
# This line should be removed if the legacy DWC2 controller is required
# (e.g. for USB device mode) or if USB support is not required.
otg_mode=1

[all]

[all]
dtoverlay=gpio-fan,gpiopin=26,temp=60000

funny enough installing older bullseye solved my issue and now I am able to use camera

mryel00 commented 4 months ago

funny enough installing older bullseye solved my issue and now I am able to use camera

To clarify he is installing with Crowsnest, so on Bullseye he installed the master branch instead of the main branch.

Also worth mentioning the error appears on the first connection to one of the endpoint.

mokshmridul commented 4 months ago

I have exactly the same problem/ Have moved to spyglass for the moment, which is working perfectly.

arcadien commented 4 months ago

afaik Spyglass streams mjpeg and does not use hardware acceleration

ayufan commented 4 months ago

@arcadien Can you link me to this camera? I will re-test this on latest bookworm. Is this 32 bit or 64 bit?

mryel00 commented 4 months ago

afaik Spyglass streams mjpeg and does not use hardware acceleration

Spyglass uses the HW encoder for mjpg, but only the implementation provided by Picamera2.

arcadien commented 4 months ago

It is a 64 bit Raspi:

Linux klipper 6.1.0-rpi8-rpi-v8 #1 SMP PREEMPT Debian 1:6.1.73-1+rpt1 (2024-01-25) aarch64 GNU/Linux

What information do you need about the cam? It is this one

ayufan commented 4 months ago

Its good. I have equivalent ArduCam I believe.

ss8688 commented 4 months ago

I have seem problem

arcadien commented 4 months ago

I have seem problem

What do you mean @ss8688 ?

arcadien commented 4 months ago

@ayufan the error seems to be the one fixeds in #140. Trying it

arcadien commented 4 months ago

Confirmed : cherry-picked the commit and now i have my image